Buy

Books
Click images for more details

Twitter
Support

 

Recent posts
Recent comments
Currently discussing
Links

A few sites I've stumbled across recently....

Powered by Squarespace

Discussion > Do computer models provide 'evidence'?

On the thread Matt Ridley on the ecocorporation there was some discussion of whether or not models can provide 'evidence'. I don't think I have fully understood things here. But I think the point is important so I'd be grateful for any help in understanding the issue.

People (eg Professor Kelly) have taken exception to climate 'scientists' talking about runs of a computer model as 'experiments'. I'd always taken it for granted that a model is an illustration of a hypothesis, not something that provides evidence about the physical world. That seems to agree with Professor Kelly's view.

Matt Ridley is reported as saying:
"The evidence from computer modelling? The phrase is an oxymoron. A model cannot, by definition, provide evidence: it can provide a prediction to test against real evidence."


So I was interested when Anon commented:

"I thought Ridley screwed up in this article. Of course a computer model can provide evidence, if it has been proved against reality over and over again. It is true that a model will never provide overwhelming evidence, no matter how successful it has been, but to declare that it cannot, by definition, provide *any* evidence is an obviously false claim, and just makes Ridley look dogmatic. (...) "

http://bishophill.squarespace.com/blog/2014/7/7/matt-ridley-on-the-ecocorporation.html#comment20971351

I assume that Ridley was talking about the sort of evidence that we get from physical measurements or observations of things in the physical world.

If a model is validated by comparing its output against physical measurements, is not any "evidence" that it subsequently provides simply a regurgitation of the evidence that has already been extracted from reality by the physical measurements used to validate the model? I don't think it's *new* evidence that the *model* has generated.

Does this make sense? If not, what have I misunderstood?

Jul 8, 2014 at 9:48 AM | Registered CommenterMartin A

No, computer models don't provide evidence. Yes, what Ridley and you wrote makes sense. I too found the criticism weird, especially the phrase 'Ridley screwed up'. 'I don't understand …' or 'I would have put it like this …' maybe. But 'screwed up' was totally out of order, as I tried to say right away.

Jul 8, 2014 at 10:00 AM | Registered CommenterRichard Drake

Computer models provide evidence in the same way as clairvoyants and fortune-tellers do.

Jul 8, 2014 at 10:11 AM | Unregistered CommenterRoss Lea

Computer models can provide guidance as to what is likely to happen. The accuracy of the prediction is down to how tightly constrained the problem is and how well the relationship between inputs and the result is understood.

An example is the simulation of the airflow over a wing, lift from a wing or potential interference between two tracks on a PCB knowing the characteristics of the agressor signal. All fairly well understood, constrained problems. Having said that the engineer who relies blindly on what the models say might happen is doomed to failure.

Jul 8, 2014 at 11:13 AM | Unregistered CommenterNial

I should have added the mechanism of 'climate' is so badly understood the 'models' are almost completely worthless.

Jul 8, 2014 at 11:15 AM | Unregistered CommenterNial

I should also have added that although the examples I gave above are relatively simple, the software/ algorithms to solve the problems are really fairly complex if you want an accurate prediction (complex enough that companies make good money writing it).

Jul 8, 2014 at 11:18 AM | Unregistered CommenterNial

Jul 8, 2014 at 11:13 AM | Unregistered CommenterNial

I doubt that anyone would deny that some models can provide excellent predictions - in situations where the dynamics of the system are well understood and not subject to significant random variation nor to chaotic effects. As an example, the orbit of a satellite can be modelled with precision, so that an orbital model can be used to predict with high accuracy where the satellite will be, say, 24 hours from now.

But I don't think such predictions are the same thing as providing "evidence".


On climate models, I would not have said myself that they are 'almost completely worthless'. I think they have a negative value (as potentially with any unvalidated model whose output you believe and act on) so they are worse than merely worthless.

Jul 8, 2014 at 11:44 AM | Registered CommenterMartin A

They not only provide evidence, when their output is considered with the (very inferior) physical measurements they can provide evidence of things the programmer never even thought of. As a recent example, it was the mismatch between computed model temps and imputed reality which gave the clue to all that heat which is going into the ocean. Which climate science was not hitherto aware of.

Jul 8, 2014 at 11:53 AM | Registered Commenterrhoda

> But I don't think such predictions are the same thing as providing "evidence".

No, in most engineering applications you need to test your wing or PCB to get the 'evidence' that it's going to do what you expect it to.

Jul 8, 2014 at 12:09 PM | Unregistered CommenterNial

Climate models (GCMs) do indeed provide evidence.

That the climate is not sufficiently understood to enable its future behaviour to be predicted reliably.

Jul 8, 2014 at 12:30 PM | Unregistered CommenterBig Oil

Rhoda, does that mean that evidence of ignorance is the same as ignorance of evidence? :)

More seriously, did Trenberth re-examine the output from a previous model run and find the missing heat skulking at the bottom of the oceans, or was it just assumed to be there? If the former is the case, I wonder if a description been given of the path taken by the heat.

Having said that, I have myself used molecular modelling results to lend credence to the probable structure of a class of compounds. In the publication I think I used a phrase like "in support of this, a molecular model...", not "evidence" . I would describe it as a similar situation to the engineering cases described above in that it confirmed what I already hoped for/expected for other reasons. The real evidence came later.

Some of the models also provided something somewhat 'unexpected' in that the calculations reached an interesting unusual state where the programs found it very difficult to finally converge to an energy minimum/precise structure. As a result, what should have taken hours/days would run on for months if I let them. To this day I am not exactly sure what, if anything, that told me about either the models or the molecules.

Jul 8, 2014 at 12:50 PM | Unregistered Commentermichael hart

Jul 8, 2014 at 11:53 AM | rhoda

the clue to all that heat which is going into the ocean

Isn't that only true if;-

1. All other aspects of the model are correct (unlikely in a complex chaotic system)
2 The missing heat isn't going somewhere else
3 The measurements are accurate (not proven)
4. The heat actually is missing

?

Jul 8, 2014 at 2:38 PM | Unregistered CommenterSandyS

Sandy, maybe I wasn't sufficiently tongue-in-cheek, I was trying to slide up to the proposition that models which give the wrong answer cannot be rescued by post-hoc imaginations of some 'new' factor. It just means the models are no good and those which do not model that factor must be fixed so that they do. You can't explain away an unpredicted pause and also keep your models.

Jul 8, 2014 at 3:36 PM | Registered Commenterrhoda

I have seen various definitions of 'evidence', including:

- A thing or things helpful in forming a conclusion or judgment
- Something indicative; an outward sign
- Ground for belief or disbelief

Model output seems to fit each of these. That is not to say it is 'proof'. Just like lines of evidence in a court case may be false or may still point to the wrong verdict. Models are tools, they serve a purpose. They are not perfect and their outputs need to be interpreted and treated with caution. But that doesn't prevent them from being useful.

If you reject models, what do you propose in their place, if (although this seems unlikely) you want to understand the likely course of the climate.

Jul 8, 2014 at 8:27 PM | Unregistered CommenterRaff

rhoda
Sorry I missed the sarc tag! It did strike me as an odd post from you.

Jul 8, 2014 at 8:49 PM | Unregistered CommenterSandyS

Raff,
If tried in a Scottish court the worst case verdict the CO2 could expect when charged with the evidence of the models against reality would be Not Proven

As previous posts have pointed out current climate models don't meet the first two of your points, and are designed to confirm the belief of the modeller, your third point.

Jul 8, 2014 at 8:57 PM | Unregistered CommenterSandyS

If you reject models, what do you propose in their place, if (although this seems unlikely) you want to understand the likely course of the climate.
Jul 8, 2014 at 8:27 PM | Unregistered CommenterRaff

If I point out that something does not work, that does not put me under an obligation to propose some alternative that will work. In any case, if there are firm grounds for believing that the task is fundamentally impossible, it would not make sense for me to attempt to come up with such a thing.

It seems to be common amongst AGW believers to imply that an objection to something is not valid unless an alternative to the thing is proposed at the same time.

Jul 8, 2014 at 9:28 PM | Unregistered CommenterBig Oil

Sandy, you, and it seems Ridley, are confusing evidence with proof. Models are used everywhere in engineering and science and their output is generally used as evidence in support of some proposition or other. On its own though, such evidence is not proof of the proposition. For proof, some sort of validation (a confirmation that the model is probably correct) is necessary.

Ridley seems to have tripped himself up by using the word oxymoron, thinking that it made him look clever and in failing to use it correctly looking ignorant and foolish.

Models have their uses and if one wants to understand what might happen to future climate they seem to be an essential tool, along with studies of past climate. If you don't want to understand the likely course of the climate, then you need no models, no research, no scientists. That seems to be the position of your host and other readers. If you do want to understand climate but reject models, what do you propose in their place?

Jul 8, 2014 at 10:06 PM | Unregistered CommenterRaff

Big Oil, I didn't notice your post until after posting. But I'd put you in the category of no wanting to understnd climate from what you say.

Jul 8, 2014 at 10:08 PM | Unregistered CommenterRaff

Raff - I do not know why you assume that.

Observing that a thing is not understood or thinking that it is probably impossible to understand it are quite different things from not wanting to understand it.

Suppose the discussion were about the detailed evolution of the English language from its ancestors instead of about climate. If I pointed out that this is not understood and it is probably incapable of being understood, would you infer that I therefore don't desire to understand how English evolved?

Jul 8, 2014 at 10:50 PM | Unregistered CommenterBig Oil

Raff:

How to you get from "no models" to "no research, no scientists"?

In place of (or rather, in preference to) models, I propose scientists doing research.

Jul 9, 2014 at 1:46 AM | Unregistered Commenterjollyfarmer

Raff
Models in engineering are of well understood systems and physical properties. Despite this it is rare that the final product is actually that of the original design and modelling. As a retired test and reliability engineer I can tell you from personal experience despite all the systems put in place to ensure reliability and long life neither are guaranteed no matter how many times the models are run.

For Climate models they are dealing with a multiple input chaotic system, and as we've agreed with Entropic man there are unknown unknowns,as well as known unknowns, i'm not confusing evidence and proof what i'm saying is that climate models are so far from the actuality that they shouldn't be used for any sort of prediction or planning. They certainly shouldn't be used to gamble the nations future wellbeing on their "predictions"

Jul 9, 2014 at 6:07 AM | Unregistered CommentersandyS

rhoda +100 for the "evidence" proving the heat was missing so must be in the oceans.

Raff your quite right one could have evidence that seemed to prove something but turned out not to be proof, and indeed we do, according to climate scientists there is plenty of evidence of the existence of malign climate change, only they can't prove it because their "evidence" comes from models and fertile imaginations, not observations.

In reality however the rest of us take "evidence" to mean proof. In the case of "evidence" from models it's still not evidence, or proof, because it doesn't come from the physical world it comes from the equations and personal biasses of the modellers who put the models together. It doesn't matter whether the models are right what comes out still isn't "evidence" which is a physical thing lying around waiting to be found.

Jul 9, 2014 at 7:10 AM | Unregistered Commentergeronimo

Sandy, Geronimo, you and Ridley et. al. are just confused by the word 'evidence', but it is not a difficult word. A dictionary and normal usage will help you. Most things have evidence for and evidence against. Decisions are made on the balance of evidence. Proof is another concept entirely.

What is the first thing people do if they want to understand a system, complicated or simple? They create a model, either mental or pen and paper or in a computer. Does the model get it 100% right. Sometimes yes and sometimes no. But it helps to visualize and to predict. It is a tool.

We know that climate is changing and will continue to change. This itself is a risk for our nations' future wellbeings, yet you give me no idea of how you intend to try to assess that risk, let alone to mitigate. Please tell me what you propose in place of models. Crossed fingers?

Jolly Farmer, see the link below and the lack of any complaint from the regulars about cutting research. Oh the lucky country...
http://www.bishop-hill.net/blog/2014/6/4/hitting-back-at-scientivists.html

Jul 9, 2014 at 9:51 AM | Unregistered CommenterRaff

When there is an array of models and they all tend to show results which do not match observations, in the same direction all the time, and that array is selected according to conformance to some particular interpretation of the problem and set of assumptions, there is no indication that you have evidence of anything but wrongness. Maybe the assumptions are completely wrong, maybe they just need a little correction in some assumption or other. But all you have 'evidence' of is the prejudice of the programmers.


Arguing about what the meaning of the word 'evidence' is doesn't come into it. That's just semantics. Raff argues a straw man. What would we prefer to bad models? Good models. But we'd still know that we are looking at the prejudice of the programmer, even if he happened to be right.

Jul 9, 2014 at 10:18 AM | Registered Commenterrhoda