Models are fine. People stink.
Gasp, a topic I have a fair amount of experience with! I’ve built tons of models, simulations of all kinds of things, as a part of my profession for years now. I even have a certain fondness for them. But I have to admit most of them are useless in practice, and frankly I’m glad I haven’t built any for awhile, for the problem is not the modeling of a system but the people who use it.
Look, why build a model? It allows you the ability to test hypothetical things and designs for a tiny fraction of the cost of actually building them. With the best models, people give you accurate data, make their assumptions explicit, and tweak, tweak, tweak the damn thing until it more or less works the way you would like to see. This would be about one project in fifty. A well designed model clarifies the mind and makes it clear to as many people as possible why you choose to build something in a specific way. Transparency, repeatability, and explicit thoughts and designs are what you should communicate with a model. But I can count the projects where that was so on one hand.
But how do I put this? People are lazy, vain, pompous liars who want a model to be some kind of Magic Ju-Ju that somehow make the stupid seem brilliant. Nearly every person who has hired me to make a model wants me to lie to somebody; it’s a lie with a technological sheen, so they can be unquestioned by people who don’t have contradictory models, which is nearly always the case. At least half the time people have made it very clear to me how my getting paid was tied to how pretty a lie I was able to give them. They never wanted to go through the process to begin with; they were just ordered to urk up a model which would make it look like they were serious.
Groupthink, like we see in the climate people, I think is the norm, and no model will override that. When language comes up with a fool-proof way to prevent lying, then many, many things in human life will go much better than they do…
Ambiance 
Ennui said,
November 26, 2009 at 9:55 am
A well designed model clarifies the mind and makes it clear to as many people as possible why you choose to build something in a specific way.
Let’s take that case as an ideal type of the “model.” Even so, where all intentions are honorable, best practices are observed, etc. etc., once it’s been created, it seems that the model begins to be taken (for all intents and purposes) for the reality. Or the map gets mistaken for the territory.
Of course, you pretty much give the reason for this in the second part of your post; if a decision about some future course of action must be taken and you have a “model” that purports to describe the consequences of doing this versus that and I don’t – well, you win. In fact, there’s an old policy sciences monograph that makes an argument along these lines (that along with conveying “information,” charts and tables and so forth operate as symbols of competence, rationality, etc.).
Ultimately, the problem I see with even honest modeling (be it climate modeling, risk modeling, budget modeling or anything short of the crafting of physical engineering models) is that is gives the illusion of predictibility where none can be had. And, obviously, if someone is consciously cooking the books – they’re a plague on the land.
Ron said,
November 26, 2009 at 10:13 am
Why assume predictability is illusionary? That strikes me as an overly deterministic assumption…
Ennui said,
November 26, 2009 at 10:48 am
Why assume predictability is illusionary?
Experience.
But I think I get the joke.
1. This is a model
2. Models cannot predict the future
3. Therefore this model cannot predict that models cannot predict the future.
Like the man said – Heh. Indeed.
Ron said,
November 26, 2009 at 10:53 am
My experience indicates that plenty is predictable — no joke.
jason said,
November 26, 2009 at 7:22 pm
Ennui is right. I mean, meteorological models like ECMWF and GFS don’t help with weather prediction. The GFS ensemble simulations have never helped with predicting where a hurricane might go. All those simulations and models used by NOAA for weather forecasting are empty little computer programs politically controlled by selfishly motivated individuals who are manipulating the data so they can make a buck or further a personal agenda while the rest of us soak in the rain when we expected sunshine.
Then there are the plane models used to study aerodynamics and stealth before any physical material is even molded. Thank goodness someone knows you can’t build a test plane based on simulations. Common sense says build it and fly the damn thing. If you crash: Well, that means it doesn’t work and you have to start over. Build a new one and hope no one dies when you fly it.
And imagine the mishaps that would ensue if we ever launched a satellite without testing it first. I mean, really! Thinking you can put rovers on the moon without testing them first, thinking you can slam a rocket into the pole of the moon to find water without ever having physically tried it before. Honestly, simulations and models would be meaningless in such cases! Utter nonsense, that is. Build the bastard and blast it off and keep your fingers crossed, that’s what I say.
That’s why we visited Saturn before we sent probes to study it. That’s why the Voyoger systems were flown without any idea of what would happen–but we knew we humans would get it right without models and simulations because we’d already been through the whole solar system and beyond. That’s why the first nuclear reactor was built with blindfolds–since we’d never think of using real science to plan it ahead of time, to see what might happen, to test what we knew against what we think.
Gosh, wish I’d realized that profound truth sooner…
Ennui said,
November 26, 2009 at 9:14 pm
Jason,
Sigh. I somehow remember exempting engineering models. If you’re at all interested at what I was getting at, think about it along these lines.
There is frequently a cargo cult mentality involved in applying the tools of the hard sciences to phenomena for which they’re not appropriate. The techniques look just as rigorous and this leads people to think that they are just as rigorous. Think about the models behind the credit default swap markets.
Also, it’s interesting that you would cite weather prediction as a success story for the use of models. For all the effort put into it, it still seems amazingly squishy to me. Back in grad school I knew someone who, on a lark, created their own model for predicting precipitation in a given locale (Boston, I think). The model was as follows: if it rained today, it will rain tomorrow; if it didn’t rain today, it won’t rain tomorrow. Statistically, he beat the National Weather Service.
I really don’t trust climate science enough to think it’s a good idea to completely reorder society based on their pronouncements. Again, it’s a cargo cult situation. Yeah, they have models and computers and so forth. They have all the gear. Do they have the predictive success? What I’d like to see are definite predictions (e.g., in 2010 global mean temperature, defined as the average of the temperatures measured at the following stations at the following intervals, will be X). Then we could compare that to my null model (the temperature next year will be the same as last year) and compare the differences. Is that too much to ask? If so, why?
On the other hand, if it is already being done in as clear a way as I described above, clue me in. If the climate scientists can make accurate predictions ahead of time, then they’re not the model abusers I’m talking about. It would go a long way toward convincing everyone.
trooper york said,
November 27, 2009 at 7:12 pm
I used to love to make models when I was kid.
What a rush from that glue.
But I stopped when I got old enough to buy beer.