Discussion > Hypothesis testing in climatology
Most temperature data are a good fit to a normal distribution.
Nov 23, 2014 at 6:57 PM Entropic man
EM - do you have a reference to that? I'm interested to know more about it. Is there any reason (central limit theorem?) why is normal should be a good fit? How far out does it remain a good fit?
Martin A
EM - I'm still interested in a reference.
I quickly tried computing the empirical cumulative distribution for the Central England temperature for all August days from 1772 to 2014 and it looked significantly non-normal (visibly skewed and the with the distribution on one side tailing off a lot faster than on the other side).
Unless there is something strange about the data I happened to look at (or if my calculations had errors) then assuming normality seems to me to be a very dubious assumption for temperature data. Jumping from standard deviation to confidence limits when the actual distribution is unknown looks to me like a form of self-delusion.
Martin A
Martin A
I should probably have been more concise. A single year's data would probably fit a normal distribution, so SD and 95% confidence limits could reasonably be applied to it.
This includes a graph from Hansen et al 2012a. He includes a normal distribution (black) with a number of different years of data for comparison.
Your CET data includes the LIA and the 20th century warming. I am not surprised it is skewed. Remember that a normal distribution is expected what you take samples with uniform variation about a mean. A sample of station data for one year would fit. A population of annual averages would only show a normal distribution if climate had remained unchanged.
If your CET data does it fit a normal distribution it is probably because climate has not been uniform.
Entropic man
EM - Thanks for the Hansen et al reference.
"A population of annual averages would only show a normal distribution if climate had remained unchanged.
If your CET data does it fit a normal distribution it is probably because climate has not been uniform."
EM - I think there are a wheelbarrowload of assumptions buried in what you say there. Too many to discuss here in view of your trying to draw conclusions from a time series for which there is neither a model based on the physics of what is going on nor an empirical model (because the record is a thousand times too short to generate such a model).
But it was not "a population of annual averages" I looked at but the individual daily temperatures for every August day since 1 August 1772 until 31 August 2014.
A set of averages might indeed have a roughly normal distribution for no other reason than that's what you tend to get as the distribution of the sum when you add things together, even if the samples share a distribution which itself can be quite different from normal. But you know all that, I'm sure.
Martin A
Martin A
Your CET data is still not a normally distributed population, in the statistical sense.
Let's try an example. If you took a sample of 2000 measured heights of 40 year old men, the frequency distribution would be normal, except for a small low height tail due to the odd dwarf.
If you sampled 2000 males regardless of age, you would have aa mean offset to the right and a long leftward tail due to growth in children.
Sampling August daily temperatures since 1772, they started in the depths of the LIA. The 2014 data is from a year at the end of a 130 year warming trend.
You would expect the early years to show values to the left of the mean and later years to the right. The proportion would depend on the shape of the trend line.
Since the trend was flat for the first century and rising for the second century the early readings would be similar and the later ones increasing, I would expect the mean to be closer to the 1st century average with an asymmetric distribution and a long right tail.
Entropic man
EM - I think this discussion is a bit at cross purposes. My viewpoint is that things should not be assumed to be normal, without good reason to do so, if you are going to do calculations that depend on normality.
I think you are explaining why some things turn out to be non-normal but, although I think non-normality can usually be assumed, I'm ironically not convinced by your arguments why non-normality applies in this case.
Your CET data is still not a normally distributed population, in the statistical sense.
Well it's obviously not normally distributed - it's histogram is lopsided.
.
Let's try an example. If you took a sample of 2000 measured heights of 40 year old men, the frequency distribution would be normal, except for a small low height tail due to the odd dwarf. .
Why do you have any reason for thinking that a height distribution of 40 year olds would be normal (even with odd dwarfs excluded)?
.
If you sampled 2000 males regardless of age, you would have aa mean offset to the right and a long leftward tail due to growth in children. .
Yes, obviously.
.
Sampling August daily temperatures since 1772, they started in the depths of the LIA. The 2014 data is from a year at the end of a 130 year warming trend.You would expect the early years to show values to the left of the mean and later years to the right. The proportion would depend on the shape of the trend line.
Since the trend was flat for the first century and rising for the second century the early readings would be similar and the later ones increasing, I would expect the mean to be closer to the 1st century average with an asymmetric distribution and a long right tail. .
I don't see why including slow variations in the time series should make it non-normal.
As I said, there is no statistical model available (validated that is; anybody can dream something up) fro climate data. I think you could probably model it quite well as a time series with a component whose spectrum rose towards very low frequencies (cycles per century) and fell continually towards higher frequencies (cycles per year). This would represent LIA's and MWP's as to be expected from time to time.
I think you could model it quite well as being stationary - statistics not varying with time - apart from a cyclostationary element (statistics varying cyclically at one cycle per year). Except perhaps for the effects of the cyclostationary element, it's not obvious to me why such a time series should not be normal.
In fact, you could synthesise it by starting with normal wideband random noise and linearly filtering it, which preserves normality, to finish up with the final time series as 100% normal. (cyclostationary element possibly excepted).
Martin A
Is applying statistical methods to a subset of a subset of climate data really going to tell us anything? In the analogy given above it would be like taking the average height of 2000 male skeletons from Highgate cemetery and inferring average height of males native to and living in Sapporo for instance.
As we're in an interglacial our sub-set of data doesn't tell us a great deal about anything other than the period for which we have reliable data. Predicting 100 years into the future seems just to be reading chicken entrails to me.
SandyS
SandyS - "Is applying statistical methods to a subset of a subset of climate data really going to tell us anything?"
Well, no.
As has been pointed out many times, in particular by Doug Keenan, you can't do meaningful statistical analysis on a time series unless you have a valid statistical model for the process that generates the time series.
With 'climate science', the lack of a physical understanding of what is going on, in conjunction with records of past history which are far too short to derive empirical models, confirmed statistical models for what generates the time series are lacking. So statistical analysis cannot tell you anything worth knowing.
Martin A
Radical Rodent
Do not use the word proof in this context. There is no such animal in science.
All you can do test hypotheses against reality. If reality behaves in a way consistent your hypothesis, it survives another day. If not the hypothesis is wrong. If the wrongness is small it gives you information to improve the hypothesis. If the wrongness is large then it may be better to scrap it and start again.
Stephen Hawking once said "Science is the process of replacing hypotheses which are wrong with hypotheses which are more subtly wrong."
Entropic man
SandyS, Martin A
If statistical analysis is useless, what other criteria do you suggest I uae to distinguish between my own hypothesis that warming will reaume and Radical Rodent's null hypothesis that it will remain flat.
For example, 2014 is looking likely to be a record year for recorded global temperature. Will this be evidence to support my hypothesis? Why? If not, why not?
If warming continues for the rest of the decade how much would be needed to falsify RRs hypothesis? If this time period is too short how long would you need?
These are the type of questions statistics is used for. What is your alternative?
Entropic man
Entropic man,
I don't think you can predict for decades in the future far less a century or two out by using statistics of poor, or even vaguely good data-sets and proxies. As models haven't predicted how the climate has actually behaved, and still don't, solar behaviour models don't seem much better. From first principles if the sun halves it's energy output then the planets will suffer, if it doubles then the planets will suffer. Therefore if the energy output varies by 1% then the effects will be felt by the planets.
Imagine for a moment that mankind has had no impact on climate whatsoever, absolutely none. So could you predict accurately (within 50 years) when the next iceage will start on the data currently available; or even if there will be one?
Wait and see, and plan for cold as well as warmth, and don't forget a mega-volcano or meteor strike. Which would cause most harm to the human race and make plans for that?
For me Burns said it best,
But, Mousie, thou art no thy lane,
In proving foresight may be vain;
The best-laid schemes o’ mice an’ men
Gang aft agley,
An’lea’e us nought but grief an’ pain,
For promis’d joy!
Still thou art blest, compar’d wi’ me
The present only toucheth thee:
But, Och! I backward cast my e’e.
On prospects drear!
An’ forward, tho’ I canna see,
I guess an’ fear!
SandyS
EM
If statistical analysis is useless, what other criteria do you suggest I uae to distinguish between my own hypothesis that warming will reaume and Radical Rodent's null hypothesis that it will remain flat. .
It's the equivalent of making bets on things where there is no form to guide you. The the honest and realistic position is to say "we simply can't say".
For example, 2014 is looking likely to be a record year for recorded global temperature. Will this be evidence to support my hypothesis? Why? If not, why not?
Well obviously since it has been warming off and on since records recently began, there was a finite chance (at 1 January 2014) of it being a record year.
But I don't think I have seen your hypothesis stated in the form of alternate statistical models (as in submarine detection, where you say "here is [A] a model for shrimp noise, underwater reverberation etc; here is a model for [B] shrimp noise, underwater reverberation etc plus the acoustic output of a submarine. Given the signals we have observed we shall now compute the probabilities of [A] and of [B])
Without something like that I can't say if it's "evidence to support your hypothesis" and nor can you. Or you can *say* it, of course, but only like Phil Jones saying "it is statistically significant" when he clearly didn't have a clue.
If warming continues for the rest of the decade how much would be needed to falsify RRs hypothesis? If this time period is too short how long would you need?
Did the rodent voice a hypothesis? My recollection was that she simply said she would not be surprised if it got cooler.
These are the type of questions statistics is used for. What is your alternative?
Your question is one that is repeatedly asked by Believers in various forms. Chandra (I think it was Chandra) was repeatedly saying something here like "OK what do you propose using if you say the models are no good? Surely they are the best thing we have?".
The point I'm trying to get across is that if your preferred method of foretelling the future (GCMs, statistical analysis of temperature records, tarot cards, etc) is no better than simply guessing, it's better (and avoiding self-delusion) to simply say "We simply don't know, and we have no way of telling beyond a guess".
Can you see what I'm getting at? I don't know how to put it any clearer.
Martin A
Actually, Entropic Mann, my null hypothesis is “wait and see”. I fear temperatures will plummet; I hope temperatures amble along upwards; what I doubt they will do is soar. However, whatever they do, I suspect that, by the year 2100, they will be within 1°C of where they are, now.
You do seem to be falling into a common fault: you seem to believe that a theory or hypothesis has to be replaced by another theory or hypothesis before it can be declared void. Wrong! All that is needed for a theory or hypothesis to be void is for it to be shown to be invalid. AGW has that in spades – we do not need to go to Hans Custers to see that (and, you have to admit, if I can fisk him (if that is the right term) as I did on Unthreaded, then his arguments are seriously flawed).
Radical Rodent
Entropic Man started the thread with his particular version of hypothesis testing in climatology. Since then it has wavered between the recent verification of the AGW hypothesis (in terms of surface temperatures, polar ice melt, sea level rise) and hypothesis testing in general.
Our host – Bishop Hill – has most of his posts on policy, not on climate theory. Climate policy is mostly concerned with mitigation - the reducing global CO2 (and other GHG) emissions, to constrain global warming. This, the experts believe, will avert a climate catastrophe. The test of whether policy will work is to show that changes in global emissions are related to changes in temperature. Specifically, increases in global emissions are related to increases in the global surface temperature anomaly. Conversely, if (due to a global crisis), the rate of increase in emissions slows or stalls, this show be reflected in a reduction in rate of increase in temperatures.
Since 1945, human emissions have been increasing. But the rate of increase has varied. CDIAC have estimates of global carbon emissions since 1959. I have charted the 5 year centered moving average of the year-on-year percentage increase in emissions.
Of note:-
1. Prior to the oil embargo of 1973, emissions were increasing at around 5% a year. Global Temperatures were static or falling slightly from 1945-1976.
2. After 1973, the rate of increase slowed dramatically. Around 1976 global temperatures took off.
3. In the 1990s, global emissions increases were very low. But global average temperatures were increasing at a rate of 0.2oC per decade.
4. Post 1998 the rate of global emissions increases took off, just as global temperature increases stalled.
In every single one of those four periods, emissions growth went the opposite way to global surface temperatures. That strongly suggests that reducing global emissions will not stop global warming. At best there is no empirical link.
Kevin Marshall
+1 Kevin Marshall
diogenes
Entropic, you say:
The real need is to decide where reality stands. It is somewhere between "What warming?" and "Oh shit, oh shit, oh shit, we're going to die!"
...If formal hypothesis testing is not appropriate what alternatives would you suggest?
AND then you say:
All you can do test hypotheses against reality. If reality behaves in a way consistent your hypothesis, it survives another day. If not the hypothesis is wrong.
I did not say hypothesis testing is 'not appropriate', I said it is 'impossible.' If you agree, you can say 'yes'. Or else, 'no'. There are sciences where hypothesis testing is possible and on much shorter timescales. Climate science is not one of them. Climate science is largely an observational discipline.
If you make up a hypothesis against a system exhibiting random variability and the variability generates patterns recorded in observations that merely happen to coincide with the hypothesis' predictions, you will be falsely lead to reject the null hypothesis when in fact you shouldn't have. It is called a type I error. The hypothesis will 'survive another day' when it should not.
Do you get this, or not? 'Cause I don't see any evidence in your comments that you do. You seem lost in a reverie.
If you can write a thread, give it a heading but are not able to defend it or substantiate it, what position are you in to call the people discussing issues with you "stupid"?
Secondly, could you please point me to the source of the Stephen Hawking sentence you quoted?
shub
Shub - it may have been a different Hawking.
"As usual, I tried to determine the source of the quote before using it."
An excellent principle to apply. EM... Pay attention!
Martin A
SandyS, MartinA
I am aghast!
Entropic man
Excellent find, Martin. Notice even the quoted text doesn't quite match EM's involuntarily manslaughtered version attributed to Hawking.
shub
Entropic Mann: you object to my choice of words: when I use the word “proof” that is because it is proof that I mean, nothing more, nothing less (listen to the caterpillar); a black swan proves that not all swans are white – i.e. it presents irrefutable evidence. That you disclaim that there is no such thing as proof in science does indicate a flaw in your thinking; is the black swan not proof? Or do you subscribe to the idea of it being a white swan in a different guise? It might be difficult to prove a theory, which is why it generally holds until it is proved to be wrong. Okay, metaphorical black swans might be rare in science, but that does not preclude their existence.
If reality behaves in a way consistent your hypothesis, it survives another day.But, if reality does NOT behave in a way consistent with your hypothesis, your hypothesis is wrong, and should be discarded; it does not NEED to be replaced with another hypothesis. Or perhaps you think that the “gravity waves” hypothesis of a few years ago must still hold sway, despite being comprehensively shown to be wrong, as there has been no other hypothesis to replace it?
Climate “scientists” are working with a huge, complex, chaotic system, one that cannot be replicated in a laboratory. Sure, they can replicate parts of it; they can show that CO2 can absorb infrared radiation – who knows, they might even be able to show that radon molecules can play badminton; however, out “in the wild”, the shuttlecock will soon get lost, so “wild” radon may never play the game. Yes, continue the study of the climate system; but it will take decades of intense observations to get any meaningful results, so do not base governmental policy – or even global policy – on the very limited results we have, so far. Also, by giving the impression that all the results are in and the solution has been found, there is the very real danger that no further meaningful work will be done – after all, if the science is settled, why bother with further study?
Radical Rodent
"Notice even the quoted text doesn't quite match EM's involuntarily manslaughtered version attributed to Hawking."
No. 'hypothesis' (™ Entropic Man) was the give-away.
Martin A
Entropic man
Why should you be aghast?
We are discussing models with acknowledged short comings based on incomplete data and not fully understood mechanisms; for instance energy suddenly transferring to deep ocean heating from atmospheric heating. Why would you expect them to be any better than a guess? Given enough guesses one is going to be correct or pretty close to.
BTW I am pretty optimistic about mankind's survival, just looking to the future and fearing we've guessed the wrong scenario, in which case there's trouble ahead. My own guess is that it will turn colder in the medium term and who knows after that.
SandyS
Martin A
It is from a Brief History of Time. The exact wording is:-
"Progress in science consists in replacing a theory which is wrong with one that is more subtly wrong."
Entropic man
Radical Rodent
Black swans DISPROVED the hypothesis that "All swans are white."
Proof and disproof are not the same. You can never prove that a hypothesis is correct, because there is always the possibility that the next experiment will prove you wrong. You can disprove a hypothesis by demonstrating that its predictions are different from reality. You then modify it if the difference is small or scrap it if the difference is large.
This is why the type of sceptic which insists that the onus of proof is entirely on the scientists sounds so foolish to the scientists.
Entropic man
Shub
We are drifting into the philosophical terrritory of the debate between Fisher and Pearson in the 1930s.
The real need is to decide where reality stands. It is somewhere between "What warming?" and "Oh shit, oh shit, oh shit, we're going to die!"
What techniques should be used to analyse the available data. If formal hypothesis testing is not appropriate what alternatives would you suggest?
For a scientist "I don't know" is a perfectly acceptable statement. Politicians making policy have no such luxury. What techniques should be used to derive the policy advice they demand?
Nial
I don't take the deviation as proof that global warming has stopped. I am using the to get a feel for the difference between internal variation and external variation. I am not seeking proof, just an indication which way to bet.
CO2 was 280ppm in 1880 and 315ppm in the mid-1950s. It increased by 35ppm in the period whose centre you regard as a control with no change in CO2. This is why I suggested to shub that 1850-1880 ,with no change in CO2, would be a better choice for a control.
The full question should be "Why did 1910-1940 and 1970-2000 show warming while 1940-1970 and 2000-2014 did not?". The sixty year AMO cycles enough heat to change surface temperature up or down by 0.15C from the trend and would suffice to cancel or enhance warming as observed.
My own hypothesis is that tthere has been s a 0.6C/ century warming trend since 1880 due to increasing CO2. Superimposed on that is a +/- 0.1C internal variation, a +/-0.15C variation over a 60 year cycle due to AMO, and larger variations due to aerosols, inaolation etc.
By definition I can't prove it, but it fits the data and I have not falsified it so far.