I wasn’t going to post anything about the CRU emails story (apart from my attempt at humour), because I think it’s a non-story. I’ve read a few of the emails, and it looks no different to the studies I’ve done of how climate science works. It’s messy. It involves lots of observational data sets, many of which contain errors that have to be corrected. Luckily, we have a process for that – it’s called science. It’s carried out by a very large number of people, any of whom might make mistakes. But it’s self-correcting, because they all review each other’s work, and one of the best ways to get noticed as a scientist is to identify and correct a problem in someone else’s work. There is, of course, a weakness in the way such corrections are usually done to a previously published paper. The corrections appear as letters and other papers in various journals, and don’t connect up well to the original paper. Which means you have to know an area well to understand which papers have stood the test of time and which have been discredited. Outsiders to a particular subfield won’t be able to tell the difference. They’ll also have a tendency to seize on what they think are errors, but actually have already been addressed in the literature.

If you want all the gory details about the CRU emails, read the RealClimate posts (here and here) and the lengthy discussion threads that follow from them. Many of the scientists involved in the CRU emails comment in these threads, and the resulting picture of the complexity of the data and analysis that they have to deal with is very interesting. But of course, none of this will change anyone’s minds. If you’re convinced global warming is a huge conspiracy, you’ll just see scientists trying to circle the wagons and spin the story their way. If you’re convinced that AGW is now accepted as scientific fact, then you’ll see scientists tearing their hair out at the ignorance of their detractors. If you’re not sure about this global warming stuff, you’ll see a lively debate about the details of the science, none of which makes much sense on its own. Don’t venture in without a good map and compass. (Update: I’ve no idea who’s running this site, but it’s a brilliant deconstruction of the allegations about the CRU emails).

But one issue has arisen in many discussions about the CRU emails that touches strongly on the research we’re doing in our group. Many people have looked at the emails and concluded that if the CRU had been fully open with its data and code right from the start, there would be no issue now. This is of course, a central question in Alicia‘s research on open science. While in principle, open science is a great idea, in practice, there are many hurdles, including the fear of being “scooped”, the need to give appropriate credit, the problems of metadata definition and of data provenance, the cost of curation, and the fact that software has a very short shelf-life, and so on. For the CRU dataset at the centre of the current kerfuffle, someone would have to go back to all the data sources, and re-negotiate agreements about how the data can be used. Of course, the anti-science crowd just think that’s an excuse.

However, for climate scientists there is another problem, which is the workload involved in being open. Gavin, at RealClimate, raises this issue in response to a comment that just putting the model code online is not sufficient. For example, if someone wanted to reproduce a graph that appears in a published paper, they’ll need much more: the script files, the data sets, the parameter settings, the spinup files, and so on. Michael Tobis argues that much of this can be solved with good tools and lots of automated scripts. Which would be fine if we were talking about how to help other scientists replicate the results.

Unfortunately, in the special case of climate science, that’s not what we’re talking about. A significant factor in the reluctance of climate scientists to release code and data is to protect themselves from denial-of-service attacks. There is a very well-funded and PR-savvy campaign to discredit climate science. Most scientists just don’t understand how to respond to this. Firing off hundreds of requests to CRU to release data under the freedom of information act, despite each such request being denied for good legal reasons, is the equivalent of frivolous lawsuits. But even worse, once datasets and codes are released, it is very easy for an anti-science campaign to tie the scientists up in knots trying to respond to their attempts to poke holes in the data. If the denialists were engaged in an honest attempt to push the science ahead, this would be fine (although many scientists would still get frustrated – they are human too).

But in reality, the denialists don’t care about the science at all; their aim is a PR campaign to sow doubt in the minds of the general public. In the process, they effect a denial-of-service attack on the scientists – the scientists can’t get on with doing their science because their time is taken up responding to frivolous queries (and criticisms) about specific features of the data. And their failure to respond to each and every such query will be trumpeted as an admission that an alleged error is indeed an error. In such an environment, is it perfectly rational not to release data and code – it’s better to pull up the drawbridge and get on with the drudgery of real science in private. That way the only attacks are complaints about lack of openness. Such complaints are bothersome, but much better than the alternative.

In this case, because the science is vitally important for all of us, it’s actually in the public interest that climate scientists be allowed to withhold their data. Which is really a tragic state of affairs. The forces of anti-science have a lot to answer for.

Update: Robert Grumbine has a superb post this morning on why openness and reproducibility is intrinsically hard in this field

37 Comments

  1. wow, this post synchronistically matches up with my text in Comment #3, blank verse post from yesterday.***

    when saying “huuugely tremendous info overload” there, I was thinking more along the lines of “Mike” being a variable with many-many values, and counting, across time and genres, academic + nonacademic. Within numerous simulacra of the same honest-work-its-denigration-&possible-exoneration pattern – wherever 1 looks.

    A principled Q: Could more than 1 side (out of many) involved in each and every controversy perhaps have (a measure of) “good” intentions, and could it be that, in the final analysis, frustrated communication&collaboration result in mutually defeating Science Wars – unconstrained to the “sciences”, not by a long shot!

    ——–
    ***To de-poke some of Comment #3 in view of what I say here: the post *is* black-humour funny, not an “attempt @”. Tx – no SE needs a thorn wreathe of glory:)

  2. Pingback: CRUde Hack, everybody loves a charade « Greenfyre’s

  3. [comment withheld pending citations]

    [Vadim – I’ve held your post for now. You make some vague allegations about software errors, but provide no details or links. You also allege that there was no review process, but offer no evidence for this. And you’re going to have to demonstrate that the software in question actually affects some specific published science for me to pay any attention. Come back when you have done your homework. And please use a full name rather than a pseudonym. – Steve]

  4. Hi Steve,

    I strongly disagree that “because the science is vitally important for all of us, it’s actually in the public interest that climate scientists be allowed to withhold their data.”

    You offer several reasons for this “tragic state of affairs.” However, the reasoning is based on me adopting an unacceptable assumption. The ASSUMPTION that the climate science based on the data is correct. This is circular reasoning. The climate science is that which the data confirms or falsifies. How else can we know the science actually is “vitally important?” The data.

    The only foundation for the science is the data. Calling for the scientific method to be short-circuited for some expediency, no matter how practical sounding, is, IMHO, doomed to ultimate failure.

    I realize there is more than just the science at stake. But even if your political or environmental convictions about climate change are vitally important to you, I suggest just dropping this science and resting your advocacy on the significant body of other climate science that does have the data available.

    Finally, I found your “denial of service” analogy clever/creative. Using your analogy, you seem to be saying that the best way to confront these “denial of service attacks” is to take the climate science quality assurance “servers” “off-line”. They have important climate science “programs” running and don’t have the “clock-cycles” to spare.

    Although I would agree that quality assurance is a different activity, with a different purpose, than creating quality, I believe it is an absolutely necessary activity nonetheless. Something the climate scientists cannot avoid. The philosophy of science provides the “denialists of service” with the right/duty to be skeptical “attackers.” So the best plan is to bring more “servers” online.

    George

    [George – thanks; interesting points. I’m not fully convinced it’s in the public interest for science to be done behind closed doors either. But I’m at a loss to figure out how to deal with the denial of service attacks. Bringing more “servers” online would be great, but would require a lot more funding, and a supply of new PhDs that just does not exist.]

  5. Bottom line: Denial of Service is Service of Denial.

  6. You remind me of a debate/discussion I had at a data-oriented meeting. It was put forth that if all the data were archived, with flags as to whether it had been used or not, then this was enough for any later (envisioning years-decades later) scientist to reconstruct the derived products (it being assumed that journal article descriptions of algorithms sufficed). I … er … was not persuaded.

    Given my efforts to reproduce my own results years after the fact, I already could see it ludicrously impossible that you could reproduce results given only data flags and journal descriptions. Even given the exact same input data, and the exact same programs (actual programs, not merely a journal description of what the program ‘should’ look like) it isn’t trivial. Different, perfectly reasonable, interpretations of what the scripts should look like would give different results. Even given the exact same scripts _and_ programs (source) _and_ data the results needn’t be the same. After all, we’ve linked the program using math libraries, and the math library today is not _exactly_ the library of 5, 10, 20 years ago. And (we’re discovering the hard way at work) those library differences can show up as important.

    Then you open the MPI kettle of worms, and the results aren’t even reproducible — exactly — _today_ using exactly the same scripts, programs, data, and libraries. Because the asynchronously passed messages didn’t pass in the same order from run to run, runs 10 minutes apart. (Again, a hard-way discovery at work.)

    All this concerns me, quite irrespective of anything to do with CRU. Because the ideal of open-ness is moved many steps farther away when the same person who wrote ‘everything’, and has exactly the same data today as N years ago (because it’s his own archive from back then), doesn’t easily get the same results as before. (I _do_ get the same results on that system, but I have to put some work in to it. And it isn’t _that_ big a system, low Tb for data, a handful of scripts and program, none of which are multi-processor. But it’s also me, knowing everything I know, trying to reproduce my own work of 5-15 years ago.)

    So I look for the computer scientists to help with pointers on how to do the software engineering. Hopefully CS folks who are willing to do some translation to us folks who are just scientists, vs. being software engineers. Now that I’m leading a group effort on another data problem, I’m feeling this keenly. Irrespective of other people duplicating our work, I’d like for each of us to be able to duplicate each others’ results. Much the same (a software engineering friend in college persuaded me) techniques are involved either way.

  7. Robert Grumbine — Being a non-scientist, your comment (and this excellent post by Steve) worries me. What if climate scientists made all their data and codes available, as all the sceptics are calling for? Wouldn’t the sceptics simply run them all and find — surprise, surprise — the results are different than what the climate scientists came up with? As you point out, it’s almost impossible to get the same results if you’re not the guy who did the original runs. The potential for sceptics to obsess over slight differences increases the volume of mischief-making exponentially.

  8. @Robert Grumbine

    As a software geek, I am sorry to say that AFAIK, the current state-of-the-art is primitive. For example, at the Yucca Mountain Nuclear Waste Repository, all the safety-related scientific analyses for storing the waste over the centuries (for water infiltration, climate change, volcanoes, earthquakes, etc.) contained a unique tracking number for both the software configurations and the data sets used. A commercial software configuration control application was used for the programs and operating enviornments, and a custom data set tracking application was used for data. (Yes, even if you just used the Stefan-Boltzmann constant in an analysis its value must be stored as part of the unique data set for that analysis. It could not simply be referenced.) Thus, for any scientific conclusion presented in any analysis, the exact software configuration and data set(s) were identified so that they could at some arbitrarily later time be confidently extracted from the respective databases.

    For which safety-related software/data was this not entirely possible to accomplish? You would be correct: MPI applications.

    A know of similar approaches taken in other rigorous software quality assurance areas.

    George

  9. I think you’re heading in the wrong direction on this. i’ve been reading climate audit for about 3 years, and it is clear to anybody with the slightest bit of common sense that some of McIntyre’s points are undeniably true. E.g. Mann et. al. did use the wrong principal components procedure; they did use data that contributes nearly all of the variance and which is known to have 6-sigma changes of non-climatic origin, etc. If you would read what has been written there (before climategate broke), you will know that corrections Steve has made or errors he has found are sometimes trivial, and sometimes invalidate entire papers.

    If you are unconvinced and not familiar with climate science, take a moment to look up where the phrases “divergence problem” and “teleconnection” came from, and then tell me that this is a serious science not hell-bent on showing warming no matter what.

    [Carl, I guess that might be true, but you’ll have to provide citations to the invalidated papers and subsequent corrections to convince me. All the independent reviews I’ve read say that McIntyre’s “corrections” are irrelevant to the conclusions of the papers he criticizes. The last bit seems to be a red herring. There’s plenty of good peer-reviewed literature on the divergence problem and teleconnections, none of which appears to have anything to do with McIntyre. – Steve]

  10. pointer:
    I think the important part is the difference between a skeptic and a faker. A skeptic can understand that scientists aren’t responsible for compilers, hardware architecture, system libraries, and the like. They’ll hold some toes to the fire as to showing that the differences are _only_ because of that kind of thing, but they’re also willing to do some work themselves to show where they think they see a result that’s outside that range.

    The fakers, on the other hand, only claim to be skeptics. They dishoner the term skeptic. As long as you get a result they don’t like, they’ll reject it and attack it and you. They’ll complain about your results, but not put the study in to see whether the issue is their compiler being different from yours vs. you having a real mistake. For them, it really doesn’t matter how much work you put in to making your systems transparent, open, reproducible, etc.. So I don’t worry about them myself. I _do_ worry about reproducible, or at least as reproducible as reasonable, data processing and modelling.

    George:
    Constants, even something as ‘obvious’ as the Stefan-Boltzmann constant, were something I got introduced to as problems in numerical analysis quite a few years ago. Folks who haven’t studied it just don’t realize that the computer doesn’t (usually) work with the number we write down. Instead using the computer representation of it. This can be close. It will be ‘as close as possible’ (according to how the compiler works). But it just isn’t exactly the same. Hence the requirement you mention for recording exactly which value of the constant you used. A wave modelling friend mentioned recently that g (acceleration due to gravity) can make for major differences in results. A usual rounding is 9.81 meters per second^2, but it can also be given as 9.8066. The difference between the two actually matters to parts of the wave models. On the other hand, there are other parameters in other models that can be varied by 50% without any noticeable changes.

    I’m hoping you’re wrong about the state on the software engineering side. I keep hoping there are much better (than we scientists use) methods already known and just waiting for us to talk to the right people. That’s often been the case.

    all:
    I’m going to be taking this up on my blog, on Monday. It’ll be a somewhat different vein, more science-oriented. I’ll still be welcoming advice on the software engineering over there as well.

  11. Robert – I’m working on a paper on the software verification and validation processes used for the more sophisticated GCMs. The processes are very interesting, and significantly more effective than commercial software engineering techniques. We also have some preliminary defect density studies that indicate climate models appear to be higher quality than most commercial software.

    Preliminary findings are in my CiSE paper (http://dx.doi.org/10.1109/MCSE.2009.193), but that paper focusses on a the general processes used, rather than the resulting code quality.

    I can confirm the issue with constants. It reminds me of a tale I heard at the Hadley centre about why it’s often very hard to integrate model components developed in different labs – in one case, a slight difference in the constant for the diameter of the earth in two different models took them ages to track down and fix.

  12. A few questions:

    1. You refer to a well-funded PR campaign. Are you specifically claiming that Steve McIntyre (the focal point) is funded by some organization and if so which?

    [I’m claiming nothing about Steve McIntyre – SME]

    2. On this basis, would funding by governments or NGO’s and charities providing alternative views be wrong?

    [Strange question. The issue isn’t alternatives views. The issue is the integrity of the scientific process. Funding that is provided for scientists to do good work, without any political agenda attached is great. Funding aimed at proving or disproving a particular proposition is not – SME]

    3. What do you believe is the ratio of funding between the two camps?

    [“Two camps”? What two camps? Are you suggesting that for every dollar we spend on science we should devote another dollar to the attempts to undermine and discredit that science? That would be perverse. – SME]

    4. You talk about a DoS attack, but from my reading of the emails this seems a highly misleading characterization. Phil Jones denied the very first request he received (from Willis Eschenbach).

    [If you read my post carefully, you’ll see the analogy with denial-of-service attacks is made with what happens *after* data is released – SME]

    5. Do you have an opinion on whether there should be higher standards of openness in areas of science that seek to influence public policy?

    [Science never seeks to influence public policy. It seeks to understand the world. If the results of that understanding lead to the need for changes in public, then good, open communication of the science is very important. My post simply points to a poisoned environment in which such openness is very difficult – SME]

  13. @Jasper Westaway

    You talk in vague terms about well-funded PR around skepticism but are you willing to make specific claims? If not then you are, to use your own phrase, merely “sowing doubt”.

    [It’s well documented in the book “Climate Coverup” among other places. I’ve deleted the rest of your post. We’re not discussing advertising budgets, we’re discussing science. If you can’t tell the difference, you’re in the wrong place here – Steve]

  14. Dear Steve and Jasper,

    I keep getting this pic of shades of grey instead of black and white – my humanitarian academic bias or genes from 2 engineers? hard to tell.

    Could you please clarify:
    1) deniers of GW?
    2) deniers of Human factor as main contributor to GW?
    3) deniers of Human factor as sole contributor to GW?

    and add at least the temporal dimension crosstabulating the above:
    i) in the past, now and forever?
    ii) from now on?
    iii) for now?

    perhaps within 2) and 3), also:
    a) CO2 mostly?
    b) CO2 only?

    Do we know for sure that if there is a “critical mass” that makes a factor worth considering as contributing to GW, the relations between “visible”/”significant” and “under the radar” factors will remain the same? Or at least, even if there are changes in the relative positioning, there will be no fluctuations in the significant vs insignificant set of “potentially contributing” factors to the point of migrating/switching members?
    A. for a sufficiently long period of t?
    B. now and forever – well, until GW flips into the next Ice Age?

    Thus, my earlier question about how to handle (or how much of it to handle) the computational complexity of a model with multiple variable factors – not excluding SE for GW.

    Let us say, transcending them, if you don’t mind?

  15. So…. Despite the fact I’ve not made a single claim about the truth, or otherwise, of global warming, this is what it comes down to: I have to tick a box for us to move forwards.

    The irony here is that you speak of a poisonous atmosphere and implication is that skeptics have made it poisonous. But your blog item is as poisonous as it gets. Specifically you are claiming with various degrees of explicitness:

    – AGW skepticism is a well-funded campaign (ie. it’s a conspiracy and anyone that doesn’t loudly support AGW is a toady of said conspirators)
    – The goal of requesting data and models is to shut down the process of science itself
    – People who express skepticism are anti-science

    The poison is a function of your preconceptions. You believe everything I’m saying is because I do not believe in AGW. In fact I’m saying these things because I’m interested in methodology, specifically:

    – Transparency as a trend that will improve the world – I’ve written about it many times
    – How we can make science more effective
    – And from a personal perspective – with a strong background in data modeling and forecasting – the desire to understand and reproducing models that sit at the heart of policy decisions, merely so that I can be a better informed citizen.

    But now, because I uphold those interests, you must seek to understand my perspective narrowly in terms or whether I am with you or against you.

    I somehow doubt you wish to continue this debate; if that is not the case, then I’m happy to do so BUT:

    – Please stop cutting parts of comments
    – Please answer my questions before I answer yours

    [Jasper: I fail to see how I’m creating a “poisoned atmosphere”. I’m pointing out that open science is nowhere near as easy as many people seem to think, and in particular it enables DoS attacks in situations like this. It is not correct to characterize my post as being against all forms of skepticism]

  16. Hi Steve,

    Let me be just a little more direct about why I disagree that the CRU emails are a non-story, and to perhaps further convince you to agree with me that it is not “in the public interest for science to be done behind closed doors either.”

    IMHO, at the heart of Climategate is the horrible specter of what Richard Feynman called, in his famous CalTech 1974 graduation address, “cargo cult science”. A ghost many people, including myself, sense in the CRU emails.

    What separates cargo cult science from real science? To quote from Feynman’s address:

    “It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty–a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid–not only what you think is right about it: other causes that could possibly explain your results; and things you thought of that you’ve eliminated by some other experiment, and how they worked–to make sure the other fellow can tell they have been eliminated.

    Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can–if you know anything at all wrong, or possibly wrong–to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it. There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

    In summary, the idea is to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another.”

    The CRU emails provide us with a golden opportunity to explicitly reaffirm our commitment to real scientific integrity. IMHO, a very worthwhile thing to do — for both sides of the global warming debate.

    George

  17. Jasper and George: Once again, this has nothing to do with which “camp” you’re in, nor what you believe about AGW. It’s to do with whether you respect scientific processes. If you want to demonstrate anything I’ve said is incorrect, you need to cite appropriate evidence. Unfounded accusations will be deleted. I’ve cited specific evidence to support my assertion that there is a well funded PR campaign aimed at undermining the science and sowing doubt. If there’s anything wrong with the evidence *YOU* need to do your homework and find it.

    Part of the reason why I describe it as a DoS attack is the expectation by a large number of people who have no apparent understanding of basic climatology, and who seem to be unwilling to go and get such an understanding, to get the scientists to do all the work, explaining over and over again why their analysis is correct. These scientists spent years and years becoming experts in their fields. If you can’t even be bothered to read the IPCC reports or the published papers before you criticize them, then you don’t deserve any of their time in responding to you.

    So, here’s a challenge. Which parts of the IPCC AR4 are wrong? Give me specific page numbers and citations to evidence that challenges them. If you can’t back up your allegations with specifics, you’re not welcome here.

  18. Hi Steve,

    I’m not so sure Jasper would want to be lumped in with the likes of me 🙂

    I’m sorry if you think I have been talking about IPCC AR4. I have not. Your post is about the CRU email kerfuffle and how it strengthens your belief that, because of the certain prospect of “denial of service attacks,” “it’s actually in the public interest that climate scientists be allowed to withhold their data.” I disagree strongly. I tried to explain why, but evidently failed entirely. Let me try an engineering tack.

    A few years ago Ford had a commercial that said they built quality into every car they made. They were wrong. Quality is not a property of a car. It is a state of mind, a believe, a confidence, that a person has about a car being reliable. Quality resides in the customer/manufacturer/person — not the product itself. So it can’t be built into the product.

    Rather, building quality is comprised of those activities taken to assure customer/person confidence that the product will be reliable. Quality assurance is not just testing. It is also control and documentation of the entire process of creating and maintaining the product.

    The CRU emails are affecting the public confidence in the quality of climate science in general and its data record in particular. (Do you really want me to go into the details?) And not because of anything about IPCC AR4 itself. It is in the quality of the process surrounding AR4. In particular, the loss of public confidence in that process. (Again, want details?)

    This is a serious issue because, for example, a loss of the quality of the climate data is the same thing as a loss of the climate data. This is a far greater issue than any “denial of service.”

    Anyway, that is why I believe the first step in regaining public confidence in the climate data is to present the data to the public. I am not arguing anything else.

    Frankly, I fail to see why this position would not make me welcome here. It seems specific, constructive criticism to me. But I apologize for intruding where I am not wanted.

    George

  19. META-Blog TEXT – subject to deletion
    erasing my other notes is totally OK, for familiar reasons 🙂

    Dear Steve,
    This blog has been my indisputable fav since I discovered it a couple of months ago – it is rich, really informative, and a joy for a believer in multiple perspectives, disciplinary or otherwise. Admirable, and for me, uplifting input.

    My impression is that George and Jasper are no more & no less than contributing to a thought process that is far from simple, NB! crucially, in an open brainstorming format. I’ve enjoyed the exchanges & would be happy to see more.

    one of many lynnes:)
    ————————————————————–

    I’ll go ahead and add, mega-REDUNDANTLY:

    1) Evidently, the Epistemologies AND Zones of Proximate Dev of adults (because I cannot imagine one’s ZPD can ever be reduced to zero, for anyone), despite their age and proven expertise w.r.t. A,B,C, just cannot be equivalent to/interchangeable with your own re GW or any of your other areas of expertise.

    2) You’d be totally free to teach/respond or not, I’d think, depending on how you see your, admittedly scalable, role as a scientist and professor, or simply in view of time? A Disclaimer may be in order, perhaps, or is QA a sine qua non?

    3) An interactive Blog seen as a BRAINSTORMING medium allows for good as well as lame ideas, similar or opposing approaches, the rule being that NOTHING put on the table is wrong – it is just in superposition, way before a project goal is reached.

    Or else, where’s the STORM in it that I personally have always loved, not least because, by definition, it does not entail or allow confrontation.

    Unless Serendipity is explicitly for people of your rank, say by invitation & peer review standards, it is open to WWWing – different style and commitment to any piece of writing.

    [Lynne: thanks for your thoughts. I intend Serendipity to be accessible to anyone, but I will insist that people remain constructive and on-topic. I reserve the right to delete unfounded allegations, rudeness, and repetition of denialist talking points. I’ve seen far too many climate blogs sink under the weight of nonsense in the comment stream. Hope that makes sense – Steve]

  20. I disagree with your following statement on a couple of levels: “It isn’t “skeptics” doing honest scientific enquiry that they are fending off; it’s a well funded PR campaign aimed at confusing the issue and discrediting scientists.”

    I don’t think that this is an appropriate response for information used in policy making decisions. Seriously, the potential effect of greenhouse gas emissions on climate lies somewhere between the two absolutes that I am absolutely sure are not true – greenhouse gas emissions have no effect on climate and greenhouse gas emissions are solely responsible for every climatic anomaly ascribed to global warming. Therefore it is very important that all the information used to set policy is of a higher standard than peer-reviewed papers. The fact that the information is not available and now it appears that there were folks who were actively blocking attempts to have it made available is a bad thing. Therefore these researchers should be providing the data and not trying to determine the motives of anyone who asks for it.

    The second point is that your statement suggests that there is only one well funded PR campaign out there trying to sway public opinion. Next time you see an ad on TV for any company claiming to care about doing something about global warming think about what they stand to gain. I have yet to see a corporate ad, mission statement or public presentation that wasn’t based on a corporate agenda, my company included. So again, the researchers should not hide behind the insinuation of your claim.

    Personally, I think their real motive is the monumental hassle of providing all the information. In my experience trying to provide every bit of information so that anyone can completely review my work doubles the effort and my analyses are nowhere near as complicated as these data sets. However, in order to tell society how far we really need to go to address greenhouse gas emission effects, these data should be provided.

    [Roger: I think you missed the point. Public policy prescriptions are not based on the work of any single group such as Jones’ group at the CRU. They are based on the IPCC assessments. The IPCC process is already of a much higher standard than peer-reviewed papers. All the data is open (http://www.ipcc-data.org/), and the process of reviewing the IPCC reports and summaries goes through endless review cycles with huge numbers of stakeholders. It’s hard to imagine what more openness is even possible for the “information used to set policy”. – Steve]

  21. George: Thanks for clarifying. I interpreted your previous post as *accusing* climate scientists of “cargo cult science”. If you’re talking about public perception, then yes, I agree it is a huge problem. But I really don’t think open data is the point here. Most climate data is open and has been for years. Take a look at the following:

    http://www.realclimate.org/index.php/archives/2009/11/wheres-the-data/
    http://www.easterbrook.ca/steve/?p=667

    Most of the people baying for more openness are completely ignorant of how much is already open, and would be completely unable to do anything useful with the data and source code anyway. Climate researchers already bend over backwards to give away their data and code to any other bone fide researchers. There really is no lack of openness in most of climate science, and there is a huge healthy process of comparing and validating models and datasets (e.g. take a look at http://cmip-pcmdi.llnl.gov/cmip5/). My personal experience confirms this: I’ve had no problem getting hold of code and data myself for my own research, and I’m not even a climate scientist.

    I don’t know how you overcome a public perception that has been steadily fed by lies about the science from rightwing thinktanks and fossil fuel lobbyists, and which has been inadequately explained in the media. Chris Mooney’s new book “Unscientific America” accurately diagnoses the problem, but he’s also at a loss to figure out how to solve it, other than calling for a new influx of scientifically trained graduates to get involved in science communication. He points out that scientists are partly to blame for being such poor communicators, but he stops short of saying that it’s their responsibility to fix it – expecting our top scientists to be both at the cutting edge of their fields *and* take on the communication role is simply not viable. The point of my essay above is to challenge people to think beyond simplistic calls for more openness, because openness per se is not the main problem, and I’m not even convinced it will make things better in this specific case.

    There are more extensive discussions relevant to the themes in this thread on Joe Romm’s blog:
    http://climateprogress.org/2009/11/27/%C2%AD-climategate-judith-curry-open-letter-to-graduate-students-young-scientists-climate-research-hacked-cru-emails/
    and on Eric Raymond’s blog:
    http://esr.ibiblio.org/?p=1473
    I got very frustrated with the ignorance about climate science on the latter thread, and may have let that frustration get the better of me in my snarkiness. But many of these people are completely unwilling to get off their butts and find out what climate science is actually about by reading existing free, open information sources.

  22. http://www.theregister.co.uk/2009/11/30/crugate_analysis/

    [An “interesting” account. I notice they cite extensively the politically-motivated Wegman report (Wegman is a partisan politician), and completely ignore the subsequent National Academy report. Kind of defeats your argument about political influence, doesn’t it. I like this bit:

    “Briffa himself apparently found being “true” to his science and his customer difficult. “I tried hard to balance the needs of the science and the IPCC, which are not always the same,” he writes,

    I can explain this if you’re interested. The reasons are fascinating…. – Steve]

  23. As a layman who has followed this topic for several years I disagree with your assessment. When people like Tim Ball, McIntyre and McKitrick didn’t agree with Mann’s hockey stick they had a right to question it and it turned out they were absolutely correct and discredited this fraud.

    […][snip – that’s enough. The National Academy of Sciences says otherwise: http://dx.doi.org/10.1038/4411032a ]

  24. I realise that this comment is going to be deleted so you are the only person that will read this.

    Taking a step back from the debate, do your really not see that there are several insulting accusations you make, and those accusations are unhelpful? ClimateAudit – the focal point for this angst – is generally populated by scientists (professional and amateur), engineers and statisticians. No one is paid to drive an agenda; people are genuinely concerned at the state of climate science and its relationship with public policy.

    While I’m sure you don’t think of yourself as being tribal, climate science has become highly tribal and your article is an example of tribalism par excellence. You clearly imply that skepticism is equivalent to being a denier and being a denier is equivalent to being anti-science. To quote you:

    “the denialists don’t care about the science at all; their aim is a PR campaign to sow doubt in the minds of the general public … The forces of anti-science have a lot to answer for. ”

    Can you really not see how scientists and engineers who have invested significant time into understanding the issue – and become or remained skeptical – could be anything other than insulted and further alienated/radicalised?

    I’m happy to continue the debate privately: [email address elided]

  25. Hi Steve,

    Perhaps a lack of decorum, your frustration and snarkiness as you described it, is a harmless manifestation of something more serious a lot of people that could be described as concerned, thoughtful, and educated are experiencing.

    A respected climate scientist made the following quote over at Andy Revkin’s blog today:

    “The absolute worst thing that humanity could do is mistake a short-term natural cooling for the absence of human-caused global warming and, in so doing, not transition as soon as economically possible from the fossil fuel age to the post-fossil fuel age.

    To make this mistake would leave a legacy of global warming for our children, grandchildren and multiple generations thereafter which they likely could not reverse, and for which they would likely not forgive us.”

    Of course, this is NOT the absolutely worse thing to do. That thing would be to become a cargo cult scientist. That would be unforgivable. Much worse than not being a scientist at all. Future generations expect us to solve problems based the scientific method and not a sense of conviction. Leave convictions to the religions. Leave the Bayesians to their priors. (And I say that as a Bayesian!)

    Goal oriented behavior has no place in science. In science, and name something more complicated or broader than climate science, it is simply too easy to make a mistake. And mistakes can be costly.

    George

    [Wait, what? The integrity of science is more important than whether the planet burns? That would be a tough call, but luckily it’s a false dilemma. – Steve]

  26. Jasper: I don’t see where I conflated “skepticism” with “denialisim”. All scientists are skeptics, that’s their job. People who start with a political motivation and try to undermine the scientific process are denialists.

    Let me make an analogy. Sometimes a student comes to me enraged because they put a huge amount of effort into an assignment, and got a poor mark. The amount of effort is irrelevant; what matters is whether that effort is productive.

    Whenever I take a look at Climateaudit, I am disgusted by the unwillingness of everyone there to actually look at the science. They don’t read the IPCC reports. They don’t read the literature. They exhibit in their comments a stunning ignorance about climate science and how it is conducted. Yet somehow they just know it’s all wrong. This is not a spectator sport for sunday morning quarterbacks. Climate science is hard. It takes years of specialist training. It’s like trying to equate the efforts of a bunch of angry fans in the stands with the work of a professional football team.

    (damn, an extended sports analogy. I didn’t mean to do that.)

  27. Years ago I did my thesis in forecasting foreign exchange rates with neural networks. I was all ready to pile into ‘science’: building and refining the neural networks etc

    My tutor made me do a couple of things:

    – He made me manually graph all my historical datasets to get a more tactile sense of what I was using to drive my models.
    – He made me compare the outputs of my neural networks with traditional forecasting techniques

    Doing the first I realised that there were mistakes in the data. Doing the latter I realised the science was a minor issue in the problem. Understanding what constituted ‘better’ was a harder, more significant issue – but a statistical rather than empirical question.

    I started reading Climate Audit when NASA issued figures showing a startling monthly temperature increase. ClimateAudit quickly pointed out that they they had reused the previous months data (http://www.climateaudit.org/?p=4318).

    I was intrigued and continued reading CA. It turns out that this kind of house keeping error is common place in the field. For example, station data wrongly located (urban vs rural).

    When you get into the modeling side, the statistical side is also open to debate about best practice. What can look like nitpicking arguments on CA can be rather important and subtle arguments about how sensitive models are.

    These debates resonate with my experience academically and professionally. The primary contributors to CA work extremely hard to be open and honest. People read it because it resonates integrity even if sometimes it can be tribal.

    Conversely, CA’s main targets display patterns I simply don’t recognise as exemplary of best practice.

    I’ve never contributed to CA, but I do believe they do an important job that has earned my loyalty to the extent that I will defend them against what are clearly wild claims.

    You have a rather ivory tower view of science: IPCC isn’t political but Wegman is; I’d suggest that they are all highly political, but that doesn’t make any of them wrong per se. It just means that one has to be careful. I suspect it would be truer to say that Wegman is the wrong sort of politics for you.

    [If you keep up with the scientific literature, you’ll see that the CA folks have a vastly over-inflated sense of their own role in correcting the occasional mistakes that occur in data analysis (science is self-correcting anyway), and, far worse in my opinion, spend most of their time claiming that a few minor corrections undermine all the work of thousands of scientists. Science is messy. That doesn’t make it wrong. BTW I’d recommend Bolin’s book http://www.cambridge.org/uk/catalogue/catalogue.asp?isbn=9780521880824 if you really want to find out how the IPCC separates the politics from the science – Steve]

  28. Steve & guys, I love today's evelopments! Tx!

    To your: #18 [… I've seen far too many climate blogs sink under the weight of nonsense in the comment stream. Hope that makes sense – Steve]
    Of course, and all other contributors seem to see that, in view of the restrained “I realize this post will be deleted” & similar. I’d say, the guys Respect who you are and your Time. Should they be getting Censorship for how they express themselves, i.e. what you think they mean? Cf. “the medium is the message”

    Outstanding:
    Do cut the guys some slack, for their best efforts, by the looks of it – or announce “Beware: only senior academics welcome”.
    Safe bet: ANY other top SEer would DIFFER more than you may like, OR just the way you like it? – if they are a GOOD SCIENTIST, that is.

    PESKY THOUGHT: Isn’t a MISCELLANEOUS enviro sort of helpful for Serendipity? Besides, neither side would have time for email/off-blog discussions that would do justice to respective axiologies/epistemologies, right? Imagine ME asking for an appt!?

    As it is, you have to go on a per-case basis, which would seem not necessarily principled/fair, IS WASTEFUL, and is predicated on a Foucauldian power imbalance:
    == you can delete,
    == we cannot delete, not even our own,
    == nor can we even edit embarrassing mistakes. Ouch!
    == i'm not even sure if you always see the text before it posts or when you choose to?

    To justify your – can’t call it anything nobler than – “cavalier editing” of work that is
    == not your Ss’,
    == not an article under your editorship,
    == certainly not sth you’re reviewing,
    I’d be VERY happy to see you SPELL YOUR RULES, which – granted, for a good reason – differ from those of blogs in general. Then proceed with the benefit of consistency & economy?

    OFFER w/ my very best wishes, pre-Xmas chocolatey if you wish, YOU CHOOSE:
    1) == Do not publish at all/Delete. Boy, did you have a reason to think twice before posting this one, but I like the affect it is generating, and I meant it. :):):)

    2) == Copy & paste outside of meta tag/Leave as is:
    [Thanks, guys, for being explicit, fair – and for caring about GW enough to insist that others listen to you. Looking forward to more of your exchanges, not that you have to top the current record for Every SteveE post! Also Super 2C SE worthy of SE.
    All yours et encore,

    just another lynne]

    Hope they/you like the 2 arrangements of the All Yours hit

    3. == What the heck, if u wish, publish the whole thing – for the record – learning about acad blogging boundaries Ha!

    >

    Thanks, guys, for being explicit, fair – and for caring about GW enough to insist that others listen to you. Looking forward to more of your exchanges, not that you have to top the current record for Every SteveE post! Also Super 2C SE worthy of SE.
    All yours et encore,

    just another lynne 🙂

    [Lynne – I’ll get round to writing an explicit moderation policy one day. How about this for a brief version: I’m aiming for something between a traditional “letters to the editor” section in a newspaper (extremely selective) and the the “anything goes” of unmoderated blogs. It’s probably not a sustainable policy though, if traffic continues to rise – Steve]

  29. Thanks, guys, for being explicit, fair – and for caring about GW enough to insist that others listen to you. Looking forward to more of your exchanges, not that you have to top the current record for Every SteveE post!

    Researching academic blogging – so tx for enduring the non-expert test, esp. Steve – close to LOLing

    All yoursand again,

    lynne 🙂

  30. @RogerC
    I think this episode will show that you are over-estimating the availability of the data. The CRU data set is grid point values over time. The raw data used to generate those values is the missing piece that I am very sure is not available. Given the importance of the historical record of data that is necessary.

  31. First thank you for having a civil discourse. Disagreement without personal attacks is very rare in this arena.

    I want to address one specific point you have made to discount the work done at Climate Audit that I have heard elsewhere. Specifically, I take exception to the one part of the statement that they “spend most of their time claiming that a few minor corrections undermine all the work of thousands of scientists”. How did you get the idea that there are thousands of scientists working on this specific topic?

    I have a BS and a MS in Meteorology and am a Certified Consulting Meteorologist. I don’t think it is common knowledge outside the field that after graduation the vast majority of us gravitate to a particular specialty. I, for example, know far more than almost any layman could care about the effect of lake effect snow on air pollution. While any meteorologist could basically understand my research, the number who could use the information and do similar work is miniscule. I believe the same is true when it comes to climate scientists whose specialty is estimating historical global temperatures.

    For starters the US Department of Labor has a Web site that lists information on salary ranges, careers, and job outlook for meteorology. http://www.bls.gov/oco/ocos051.htm that states that Atmospheric scientists held about 8,800 jobs in 2006. There is a breakdown into employment by industry and occupation for atmospheric and space scientists (with roughly the same total 8,568) that says 2% were in manufacturing, 7% in broadcasting, 42% in professional, scientific, and technical services, 9% in educational services, public and private, and 41% in government. I am confident that no one manufacturing or broadcasting would claim a specialty in historical global temperatures. I could not find a finer breakdown of the remaining broad categories so I will make a conservative wild guess that no more than 10% are in climatology. That drops the total to less than a thousand and you aren’t further refining to the number of people who are specifically working on historical paleo-climatology. However the number has to be increased to account for all the countries.

    I will keep looking for specific numbers but based on my knowledge of the field of meteorology in general I would not expect more than a thousand scientists world-wide are working on this topic and the number of those people whose work would be undermined is a fraction of that. There is no way the work of thousands of people could be affected.

    Does this parsing matter? I say yes because your implication is that the body of work is beyond reproach because there are so many people working on this issue. I don’t think that is true so I believe that auditing the raw data for greenhouse gas emissions policy decision making is necessary and appropriate.

    [I think we’re in agreement. When I say “thousands” I mean all the scientists in climatology, meteorology and related disciplines who have contributed to any aspect of our understanding of AGW. For the specific field of historical reconstructions, you’re right, it’s probably less than a hundred (actual numbers would be interesting if you find them). My point was in reference to the common assertion in denialist circles that any problem in the CRU analysis “must” mean AGW in its entirety is disproved – Steve]

  32. Jasper Westaway

    Steve,

    The point of your article was to argue against opening climate data.

    As multiple people seem to be advocating ClimateAudit and saying they are doing a useful job – and the FOIA requests are coming from contributors associated with CA – is your position on open science now evolving?

    [The point of my article was to point out a downside to open data in that it can allow a “DoS” attack. I haven’t seen any argument that says this risk isn’t real. Nor have I seen anyone suggest mechanisms by which the risk could be mitigated. So for the moment, I stand by my analysis. “multiple people seem to be” is a bit vague. Anyone specific? More importantly, has anyone with specialist knowledge changed their mind about CA? – Steve]

  33. @RogerC

    The raw data are the observations made by the various met. services around the world. The raw data is available from them, as it is their data. The vast majority of the CRU raw data (>95%? From RC I can’t find the comment that states it.) is available from GHCN, the remaining few percent is that stuff that was provided in commercial confidence (or whatever the exact term is), but should still be available from the relevant met. services. As the list of stations used in the CRUTEM3 dataset are freely available, all someone need do is collect the relevant data from GHCN, then request the remaining data from the services who run those uncovered stations.

    However, even that doesn’t all need to be done. As the data is gridded, a process could be made to grid all or some of the publicly available data and see if there are discrepancies. If doing that turns up differences, then it might be worth doing the rest.

  34. Adam (et al.): As you say, much of the data are available from the GHCN, at NCDC (http://www.ncdc.noaa.gov/ for the top level). And you can get the rest from the originating meteorological services.

    But … it isn’t necessarily free. In fact, some of it certainly is not free. The GHCN includes only the free material, so distribution is easy by just hanging out an ftp link (ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2/). The not-free material has usage and redistribution limitations so then each request to CRU, which did not originate the data, entails time on their part deciding for each request whether they are allowed — by the agreement they signed — to release data, for each different agreement. Or, which has pleased nobody, to refuse to release any of the data with any restrictions. The latter takes no time, so no need for concern about DoS-like requests. But the former does take time, which means you have to take someone off of the matter of doing more and better science and put them on to sieving the requests for what can be distributed. Which is its own denial of service, or at least denial of science, attack.

    Once the files have been distributed, there’s a different denial of service involvement. The formats are pretty ugly things. Required to be, as it is a World Meteorological Organization standard that all weather services must match. Check out the WMO site for their data standards. In some ways they’re quite good. Being understandable to someone not familiar with WMO is not one of those ways. As has been demonstrated before, once data are in hand, there is no end to the data format questions a scientist can be besieged with. I’ve been on both sides of this, though actually there was an end and fairly quickly. But that had more to do with the fact that at work I’m only permitted to go so far in responding (that’s seldom been what stopped it — usually my first response takes care of what people need). And when I’m the questioner, so far, my questions have not had to go terribly deep — I already knew enough of the cultural standards that a few more details were all I needed.

  35. Pingback: Phil Plait nails it… again. | Digital Chum

  36. Pingback: Phil Plait nails it… again. | Rationality Now

  37. “Science is self-correcting.”

    Uh, yeah, well, no. It would be more accurate to say that “Scientists correct other scientists.” Or even sometimes, “Scientists correct themselves.”

    It’s not like a paper is sitting in it’s journal saying to itself, “You know, something just doesn’t feel quite right,” and poof, it corrects itself.

    It takes people to see the mistakes and point them out. So to criticize people like McIntyre or Climate Audit for being anti-science deniers is ludicrous.

    [Science isn’t “papers”; science is a process. And, yes it is self-correcting: go read Thomas Kuhn. McIntyre is not a scientist, he’s a retired mining engineer. You don’t contribute to science by sitting on the sidelines throwing rocks at the bits that don’t fit your political worldview. You participate in science by looking at *all* the sources of evidence, developing and testing theories that explain them, and publishing your results so that others can validate them. – Steve]

  38. Steve: I’ll phrase it a little differently than you did, though to the same end. As you said, science is self-correcting. It is a process (at least what I call ‘live science’ in my note on science and consensus), and that lively activity grinds its way towards things that can be increasingly well-trusted.

    Doing science, participating in the ‘live science’ end of things, requires something a little different, to my taste, than just what you’re describing. Namely, the focus of your work, if it is natural science, has to be answering some question about nature. McIntyre, regardless of what is prior experience is, is simply not trying to understand nature and for this reason is not doing science. He’s generally upfront about this himself. Rather than understanding nature, he’s pursuing particular people and their methods. His questions are more in the vein of ‘Can Mann really have gotten the answers he did using the data he says in the way he says?’ That’s not a question of science. The scientific questions would be things like “What have global temperatures done over the last 1000 years?”, “Are tree rings reliable proxies for temperature?” (If not, under what conditions do they fail? Under what conditions might they remain usable?)

  39. Pingback: Nature on climategate » Mind of Dan

  40. Pingback: And He Blogs

  41. Pingback: AGU Day 3 part B: Data Provenance and Open Science | Serendipity

  42. Pingback: Climategate: the scandal that wasn’t » Mind of Dan

  43. Pingback: Science Story: the Making of a Sea Level Study « Global Warming Blog

  44. Steve,

    Excellent Blog, and I appreciate the way that you respond to the different posts, and try to engage them substantively.
    i have a miniscule background in Astrophysics and evolutionary biology, but have spent much of the last 10 years studying how people develop mental schemas, world-views, and how they self identify, including political ideology.
    I think that what we are seeing with both the “denialist” attacks on ACC and specifically with the hacked emails has a variety of consequences and flows from a few different sources.
    I have followed ClimateAudit and a few other denialist sites, as well as Realclimate, skepticalScience and other proponents of ACC. I have also read a few hundred of the most damning emails and comments on them by advocates on both sides.

    First off, in theory, I am sympathetic to the denialist premise – that there are often structural limitations and inadequate exploration of alternative factors in the current ways that modern science is conducted. I personally think that our understanding of intelligence and human nature suffers from these problems. I also think that the attacks on ACC from various sources and the PR campaign against it have actually made the science around climate change much more extensive and careful. if there had not been opposition to the theory of CO2 buildup causing potentially catastrophic change the science might have made more mistakes that would not have been discovered and the rigor of the science would definitely not be nearly as sophisticated and comprehensive as it is. On a personal level the last few months have led me to realize just how incredibly complicated the subject is. Climate science and certain biological sciences are clearly becoming very horizontally integrated over diverse disciplines, so it is hard for any one person or group of people to understand the full range involved in any theory.

    That said, it is also pretty clear to me that the attacks on ACC are largely coming from an ideological source and not a scientific one. All the scientists I see who dispute ACC have an apriori objection to the theory and are looking for ways to show that it is wrong. One frame of reference is a libertarian contrarian perspective that has the rogue non-comformist who is willing to challenge the status quo and show up the elite inbred scientists that are unwilling or unable to look at the truth and just accept the gospel of entrenched scientific opinion. I see Richard lintzen as falling in this vague camp. Then there are the engineer/geologists who are from a conservative mainstream culture that is leery of any government intrusion on freedom, and see ACC as an excuse for governmental or socialist creeping takeover of our way of life. Pat Michaels and Steve McIntyre and extremists like Ian Plimer fall into this general group. The reality is of course more complicated.

    I also think there is also an ideological perspective from proponents of ACC that is generally liberal and environmentally oriented and concerned about the possible consequences of ACC.
    In this particular case, I think the ideology of those opposed to ACC, which under normal circumstances might not be a terrible problem, in fact, so distorts the reasoning of those involved that it ends up being extremely destructive, and leads to the contradiction that you point to in your original post. Openness in this case can have serious negative consequences. For one thing the various attacks on ACC are in many cases mutually exclusive. So that the proponents of the various arguments should rationally be just as opposed to each others hypothesis as to the theory of ACC. But that does not really happen. Basically anyone’s proposal that is against ACC is part of the club and there is almost no attempt at making a consistent theory from those opposed or excluding hypothesis and their proponents that have clearly been proven completely untenable (i.e.Plimer).
    On top of this there is an extremely aggressive right wing political movement in the US that is using climate change science as one tool to demonize liberals and democrats as being part of this conspiracy to use government to undermine the American way of life. Because the science is so complicated it is not difficult to present very scientific sounding arguments that appear to completely invalidate ACC. I personally have encountered hundreds of people on right wing websites who are convinced that it has been absolutely proven that ACC is a hoax, but almost none have actually bothered to read anything other than the “science” from anti climate change sites. This of ocurse is a dangerous tredn n it’s own, as people are able to appear to have an abundance of information that bolsters their ideology, without any interest in checking the validity form sources outside their ideological framework. This also happens with the left in the US as well, but I think to a smaller degree.

    As I said, I have read a few hundred of the CRU emails and the commentaries and am surprised at how intelligent people have been able to twist them in ways that allowed right wing bloggers to totally distort them into something they weren’t. After reading the explanation of “hide the decline”, I am surprised that even McIntyre continues to profess that it is something nefarious. Though with him and some others on both sides the situation has become so personal that I can see it would be hard to let go of any opportunity to criticize their opponents. But the right wing bogosphere has created a whole myth filled with outright lies and wholesale exaggerations to support the idea of a huge scientific hoax.

    This has led to a polarization between proponents and critics of ACC, and has led to the situation that you are describing where having full disclosure is not in the interests of scientist engaged in the heart of the research on this issue. This has in fact led to an almost total politicization of the issue.

    The downside to this is that policy decisions based on science are almost impossible on a national level in the US, and the media has and will continue to focus on the enmity and horse race of “is it or isn’t it true”. So much of the actual science is lost as well as the concrete actions and development of policy that are occurring in both the public and private sectors. This very possibly will lead to a much more serious problem in the future if little or no concerted action is taken in the near term.

    On the positive side, there are intelligent people who are denialists and as has been pointed out McIntyre has made at least one if not more valuable contributions to correcting the science, and all the attention and focus on so many different aspects of the issue have made the science much more robust. All the controversy has made me learn much more about the issue, and the hacked emails, are likely going to have a positive effect on figuring out how to make rational judgements about how to archive and re evaluate data. I also see much more attention paid to policy and economics. There are valid “right wing” arguments regarding policy about taking a measured, i.e increasingly aggressive approach, rather than all out economic effort immediately. And policy is being more carefully thought by the academic establishment in conjunction with many industries.

    Of course, in my view the positive effects could be MUCH more so without the distorting effects of ideology on either side. In this particular case (which has some similarities to the arguments against evolution), it would be very easy for genuine skeptics (and there are a few of them) to be attempting to HELP scientists understand the complexities of the issues, and correct mistakes, and help to inform the public and policy makers that the limits of knowledge do NOT mean that no action needs to be taken. I think that Scientists like Lintzen and Pielke, while I have serious disagreements with their positions, serve a valuable purpose and are actively moving the science forward.

    I also think blogs such as this one, SkepticalScience and RealClimate play a valuable part.
    unfortunately I think it is largely drowned out by the right wing extremists, and will continue to be so until there is some major event that undermines the political value of being basically anti-science. I would guess that the internet is about 4 to one in hype against ACC at the moment (just a wild guess), but I also think that the clearing of scientists of any major wrong doing, and the complete exoneration of any scientific fraud, will soon have an impact on that imbalance.

    [Tony: thanks, very insightful comments. We (everyone!) don’t spend anywhere near enough time differentiating the various voices in this discussion, and understanding where they are coming from. That said, I disagree with the argument that folks like Pielke and Lindzen are moving the science forward. I’ve now been to hundreds of talks by the scientists who *are* moving the science forward, and it looks completely different to anything Pielke and Lindzen are obsessing over. Am happy to discuss this further – I’ll write more on this soon – Steve]

  45. Steve,

    I definitely don’t understand a lot of the science involved, so you may be right about Pielke and Lindzen. their’s sound like valid positions to me, even if I disagree with them. I guess my point is that they DO publish in peer reviewed science, so that their positions can be critically reviewed, and that that process “moves the science forward”. Certainly their views are used by ACC opponents to fuel confusion and distrust. Looking forward to your take on this.

    Pielke is actually quite interesting to me, because he considers himself to be a radical leftist, and I am very familiar with leftist ideology. yet, I think he makes some terribly flawed assumptions about motivations and the role of science, that I think are inconsistent with his stated beliefs.

  46. Pingback: Climategate: The scandal that wasn’t, Part 3 » Mind of Dan

  47. Pingback: Open Questions about Open Source for Open Science | Serendipity

  48. Pingback: My evil twin | Serendipity

  49. Pingback: Nature on climategate | Mind of Dan

  50. Pingback: Climategate: the scandal that wasn’t | Mind of Dan

Leave a Reply

Your email address will not be published. Required fields are marked *