Serendipity

My first year seminar course, PMU199 Climate Change: Software, Science and Society is up and running again this term. The course looks at the role of computational models in both the science and the societal decision-making around climate change. The students taking the course come from many different departments across arts and science, and we get to explore key concepts in a small group setting, while developing our communication skills.

As an initial exercise, this year’s cohort of students have written their first posts for the course blog (assignment: write a blog post on any aspect of climate change that interests you). Feel free to comment on their posts, but please keep it constructive – the students get a chance to revise their posts before we grade them (and if you’re curious, here’s the rubric).

Incidentally, for the course this year, I’ve adopted Andrew Dessler’s new book, Introduction to Modern Climate Change as the course text. The book was just published earlier this year, and I must say, it’s by far the best introductory book on climate science that I’ve seen. My students tell me they really like the book (despite the price), as it explains concepts simply and clearly, and they especially like the fact that it covers policy and society issues as well as the science. I really like the discussion in chapter 1 on who to believe, in which the author explains that readers ought to be skeptical of anyone writing on this topic (including himself), and then lays out some suggestions for how to decide who to believe. Oh, and I love the fact that there’s an entire chapter later in the book devoted to the idea of exponential growth.

At the CMIP5 workshop earlier this week, one of Ed Hawkins‘ charts caught my eye, because he changed how we look at model runs. We’re used to seeing climate models used to explore the range of likely global temperature responses under different future emissions scenarios, and the results presented as a graph of changing temperature over time. For example, this iconic figure from the last IPCC assessment report (click for the original figure and caption at the IPCC site):

These graphs tend to focus too much on the mean temperature response in each scenario (where ‘mean’ means ‘the multi-model mean’). I tend to think the variance is more interesting – both within each scenario (showing differences in the various CMIP3 models on the same scenarios), and across the different scenarios (showing how our future is likely to be affected by the energy choices implicit in each scenario). A few months ago, I blogged about the analysis that Hawkins and Sutton did on these variabilities, to explore how the different sources of uncertainty change as you move from near term to long term. The analysis shows that in the first few decades, the differences in the models dominate (which doesn’t bode well for decadal forecasting – the models are all over the place). But by the end of the century, the differences between the emissions scenarios dominates (i.e. the spread of projections from the different scenarios is significantly bigger than the  disagreements between models). Ed presented an update on this analysis for the CMIP5 models this week, which looks very similar.

But here’s the new thing that caught my eye: Ed included a graph of temperature responses tipped on its side, to answer a different question: how soon will the global temperature exceed the policymaker’s adopted “dangerous” threshold of 2°C, under each emissions scenario. And, again, how big is the uncertainty? This idea was used in a paper last year by Joshi et. al., entitled Projections of when temperature change will exceed 2 °C above pre-industrial levels. Here’s their figure 1:

Figure 1 from Joshi et al, 2011

By putting the dates on the Y-axis and temperatures on the X-axis, and cutting off the graph at 2°C, we get a whole new perspective on what the models runs are telling us. For example, it’s now easy to see that in all these scenarios, we pass the 2°C threshold well before the end of the century (whereas the IPCC graph above completely obscures this point), and under the higher emissions scenarios, we get to 3°C by the end of the century.

A wonderful example of how much difference the choice of presentation makes. I guess I should mention, however, that the idea of a 2°C threshold is completely arbitrary. I’ve asked many different scientists where the idea came from, and they all suggest it’s something the policymakers dreamt up, rather than anything arising out of scientific analysis. The full story is available in Randalls, 2011, “History of the 2°C climate target”.

This week, I’m featuring some of the best blog posts written by the students on my first year undergraduate course, PMU199 Climate Change: Software, Science and Society. The first is by Terry, and it first appeared on the course blog on January 28.

A couple of weeks ago, Professor Steve was talking about the extra energy that we are adding to the earth system during one of our sessions (and on his blog). He showed us this chart from the last IPCC report in 2007 that summarizes the various radiative forces from different sources:

Notice how aerosols account for most of the negative radiative forcing. But what are aerosols? What is their direct effect, their contribution in the cloud albedo effect, and do they have any other impact?

More »

I’ve been meaning to do this calculation for ages, and finally had an excuse today, as I need it for the first year course I’m teaching on climate change. The question is: how much energy are we currently adding to the earth system due to all those greenhouse gases we’ve added to the atmosphere?

In the literature, the key concept is anthropogenic forcing, by which is meant the extent to which human activities are affecting the energy balance of the earth. When the Earth’s climate is stable, it’s because the planet is in radiative balance, meaning the incoming radiation from the sun and the outgoing radiation from the earth back into space are equal. A planet that’s in radiative balance will generally stay at the same (average) temperature because it’s not gaining or losing energy. If we force it out of balance, then the global average temperature will change. Physicists express radiative forcing in Watts per square meter (W/m²), meaning the number of extra watts of power for each square meter of the earth’s surface. Figure 2.4 from the last IPCC report summarizes the various radiative forcings from different sources:

If you add up the radiative forcing from greenhouse gases, you get a little over 2.5 W/m². Of course, you also have to subtract the negative forcings from clouds and aerosols (tiny particles of pollution, such as sulpur dioxide), as these have a cooling effect because they block some of the incoming radiation from the sun. So we can look at the forcing that’s just due to greenhouse gases (about 2.5 W/m²), or we can look at the total net anthropogenic forcing that takes into account all the different effects (which is about 1.6 W/m²).

The problem I have with these numbers is that they don’t mean much to most people. Some people try to explain it by asking people to imagine an extra 2 watt light bulb (the kind you get in Christmas lights) over each square meter of the planet, which is on continuously day and night. But I don’t think this really helps much, as most people (including me) do not have a good intuition for how many square meters the surface of Earth has. According to wikipedia, the Earth’s surface is 510 million square kilometers, which is 510 trillion square meters.

So, do the maths, that gives us about 1,200 trillion Watts (1.2 petaWatt) for just the anthropogenic greenhouse gases, or about three quarters of a petaWatt overall if we include the cooling effect of clouds and aerosols.

But how big is a petaWatt? A petaWatt is 10¹⁵ watts. Wikipedia tells us that the average total global power consumption of the human world in 2010 was about 16 teraWatts (1 petaWatt = 1000 teraWatts). So, human energy consumption is dwarfed by the extra energy absorbed by the planet due to climate change: the planet gets about 50 Watts of extra power from climate change for each 1 Watt of power that humans actually use.

Note: Before anyone complains, I’ve deliberately conflated energy and power above, because the difference doesn’t really matter for my main point. Power is work per unit of time, and is measured in Watts; Energy is better expressed in joules, calories, or kiloWatt hours (kWh). To be technically correct, I should say that the earth is getting about 750 teraWatt hours of energy per hour due to anthropogenic climate change, and humans use about 16 teraWatt hours of energy per hour. The ratio is still about 50.

Out of interest, you can also convert it to calories. 1kWh is about 0.8 million calories. So, we’re force-feeding the earth about 6 x 10²⁰ (600,000,000,000,000,000,000) calories every hour. Yikes.

I went to a talk yesterday by Mark Pagani (Yale University), on the role of methane hydrates in the Paleocene-Eocene Thermal Maximum (PETM). The talk was focussed on how to explain the dramatic warming seen at the end of the Paleocene, 56 million years ago. During the Paleocene, the world was already much warmer than it is today (by around 5°C), and had been ice free for millions of years. But at the end of the Paleocene, the tempature shot up by at least another 5°C, over the course of a few thousand years, giving us a world with palm trees and crocodiles in the arctic, and this “thermal maximum” lasted around 100,000 years. The era brought a dramatic reduction in animal body size (although note: the dinosaurs had already been wiped out at the beginning of the Paleocene), and saw the emergence of small mammals.

But what explains the dramatic warming? The story is fascinating, involving many different lines of evidence, and I doubt I can do it justice without a lot more background reading. I’ll do a brief summary here, as I want to go on to talk about something that came up in the questions about climate sensitivity.

First, we know that the warming at the PETM coincided with a massive influx of carbon, and the fossil record shows a significant shift in carbon isotopes, so it was a new and different source of carbon. The resulting increase in CO2 warmed the planet in the way we would expect. But where did the carbon come from? The dominant hypothesis has been that it came from a sudden melting of undersea methane hydrates, triggered by tectonic shifts. But Mark explained that this hypothesis doesn’t add up, because there isn’t enough carbon to account for the observed shift in carbon isotopes, and it also requires a very high value for climate sensitivity (in the range 9-11°C), which is inconsistent with the IPCC estimates of 2-4.5ºC. Some have argued this is evidence that climate sensitivity really is much higher, or perhaps that our models are missing some significant amplifiers of warming (see for instance, the 2008 paper by Zeebe et al., which caused a ruckus in the media). But, as Mark pointed out, this really misses the key point. If the numbers are inconsistent with all the other evidence about climate sensitivity, then it’s more likely that the methane hydrates hypothesis itself is wrong. Mark’s preferred explanation is a melting of the antarctic permafrost, caused by a shift in orbital cycles, and indeed he demonstrates that the orbital pattern leads to similar spikes (of decreasing amplitude) throughout the Eocene. Prior to the PETM, Antarctica would have been ice free for so long that a substantial permafrost would have built up, and even conservative estimates based on today’s permafrost in the sub-arctic regions would have enough carbon to explain the observed changes. (Mark has a paper on this coming out soon).

That was very interesting, but for me the most interesting part was in the discussion at the end of the talk. Mark had used the term “earth system sensitivity” instead of “climate sensitivity”, and Dick Peltier suggested he should explain the distinction for the benefit of the audience.

Mark began by pointing out that the real scientific debate about climate change (after you discount the crazies) is around the actual value of climate sensitivity, which is shorthand for the relationship between changes in atmospheric concentrations of CO2 and the resulting change in global temperature:

Key relationships in the climate system. Adapted from a flickr image by ClimateSafety (click image for the original)

The term climate sensitivity was popularized in 1979 by the Charney report, and refers to the eventual temperature response to a doubling of CO2 concentrations, taking into account fast feedbacks such as water vapour, but not the slow feedbacks such as geological changes. Charney sensitivity also assumes everything else about the earth system (e.g. ice sheets, vegetation, ocean biogeochemistry, atmospheric chemistry, aerosols, etc) is held constant. The reason the definition refers to warming per doubling of CO2 is because the radiative effect of CO2 is roughly logarithmic, so you get about he same warming each time you double atmospheric concentrations. Charney calculated climate sensitivity to be 3°C (±1.5), a value that was first worked out in the 1950′s, and hasn’t really changed, despite decades of research since then. Note: equilibrium climate sensitivity is also not the same as the transient response.

Earth System Sensitivity is then the expected change in global temperature in response to a doubling of CO2 when we do take into account all the other aspects of the earth system. This is much harder to estimate, because there is a lot more uncertainty around different kinds of interactions in the earth system. However, many scientists expect it to be higher than the Charney sensitivity, because, on balance, most of the known earth system feedbacks are positive (i.e. they amplify the basic greenhouse gas warming).

Mark put it this way: Earth System Sensitivity is like an accordion. It stretches out or contracts, depending on the current state of the earth system. For example, if you melt the arctic sea ice, this causes an amplifying feedback because white ice has a higher albedo than the dark sea water that replaces it. So if there’s a lot of ice to melt, it would increase earth system sensitivity. But if you’ve already melted all the sea ice, the effect is gone. Similarly, if the warming leads to a massive drying out and burning of vegetation, that’s another temporary amplification that will cease once you’ve burned off most of the forests. If you start the doubling in a warmer world, in which these feedbacks are no longer available, earth system sensitivity might be lower.

The key point is that, unlike Charney sensitivity, earth system sensitivity depends on where you start from. In the case of the PETM, the starting point for the sudden warming was a world that was already ice free. So we shouldn’t expect the earth system sensitivity to be the same as it is in the 21st century. Which certainly complicates the job of comparing climate changes in the distant past with those of today.

But, more relevantly for current thinking about climate policy, thinking in terms of Charney sensitivity is likely to be misleading. If earth system sensitivity is significantly bigger in today’s earth system, which seems likely, then calculations of expected warming based on Charney sensitivity will underestimate the warming, and hence the underestimate the size of the necessary policy responses.

I’ve been invited to give a guest seminar to the Dynamics of Global Change core course, which is being run this year by Prof Robert Vipond, of the Munk School of Global Affairs. The course is an inter-disciplinary exploration of globalization (and especially global capitalism) as a transformative change to the world we live in. (One of the core texts is Jan Aart Scholte’s Globalization: A Critical Introduction).

My guest seminar, which I’ve titled “Climate Change as a Global Challenge“, comes near the middle of the course, among a series of different aspects of globalization, including international relations, global mortality, humanitarianism, and human security. I had to provide some readings for the students, and had an interesting time whittling it down to a manageable set (they’ll only get 1 week in which to read them). Here’s what I came up with, and some rationale for why I picked them:

1. Kartha S, Siebert CK, Mathur R, et al. A Copenhagen Prognosis: Towards a Safe Climate Future.
   I picked this as a short (12 page) overview of the latest science and policy challenges. I was going to use the much longer Copenhagen Diagnosis, but at 64 pages, I thought it was probably a bit much, and anyway, it’s missing the discussion about emissions allocations (see fig 11 of the Prognosis report), which is a nice tie in to the globalization and international politics themes of the course…
2. Rockström J, Steffen W, Noone K, et al. A Safe Operating Space for Humanity. Nature. 2009;461(7263):472–475.
   This one’s very short (4 pages) and gives a great overview of the concept of planetary boundaries. It also connects up climate change with a set of related boundary challenges. And it’s rapidly become a classic.
3. Müller P. Constructing climate knowledge with computer models. Wiley Interdisciplinary Reviews: Climate Change. 2010.
   A little long, but it’s one of the best overviews of the role of modeling in climate science that I’ve ever seen. As part of the aim of the course is to examine the theoretical perspectives and methodologies of different disciplines, I want to spend some time in the seminar talking about what’s in a climate model, and how they’re used. I picked Müller over and above another great paper, Moss et al on the next generation of scenarios, which is an excellent discussion of how scenarios are developed and used. However, I think Müller is a little more readable, and covers more aspects of the modeling process.
4. Jamieson D. The Moral and Political Challenges of Climate Change. In: Moser SC, Dilling L, eds. Creating A Climate for Change. Cambridge University Press; 2006:475-482.
   Nice short, readable piece on climate ethics, as an introduction to issues of equity and international justice…

So that’s the readings. What do you all think of my choice?

I had to sacrifice another set of readings I’d picked out on Systems Thinking and Cybernetics, for which I was hoping to use at least the first chapter of Donella Meadows’ book, because it offers another perspective on how to link up global problems and our understanding of the climate system. But that will have to wait for a future seminar…

This is brilliant:

There’s a whole series. Each video is less than three minutes, but manages to pack in some of the clearest, most informative account of climate change I’ve ever seen:

• #2: What is Climate?
• #3: Evidence of a Warming Planet
• #4: Too Much Carbon Dioxide
• #5: Where does Carbon Dioxide Come From?
• #6: Climate Change in the American West
• #7: Climate Change Around the World
• #8: Climate change in the Oceans
• #9: How Bad Could it Get?

(I’m not sure what happened to #1)

While preparing for my class this morning, I was looking for graphs that show model projections for likely warming over the coming century, and I ended up putting these two graphs side by side:

These choices are interesting for a number of reasons. Most obviously, the second graph is much scarier. The extra context from the pre-industrial era emphasizes how unusual the warming is, and the compressed timescale emphasizes the rapidity of the warming. The pre-industrial baseline shifts the Y axis slightly, so the warming from the shared scenarios, A2 and B1, looks a little worse. And by cutting the graph off at 2100, you don’t see the eventual stabilization for the B1 scenario.

The selection of which scenarios to show is important too. The SRES scenarios are projections for future emissions of greenhouse gases, based on different assumptions about economic development, globalization, and how quickly we switch to cleaner energy sources. The A scenarios represent worlds in which economic growth is emphasized over environmental protection, while the B scenarios represent a future world in which environmental measures are prioritized over economic growth. These scenarios define the emissions profiles used as input to the models, which then calculate the temperature response (because the models can only compute the earth systems’ responses to emissions levels; they can’t predict what humans will actually do!).

The choice to include A1FI in the second graph is important:

• A1FI represents strong economic growth, a strong globalization trend, and aggressive exploitation of fossil fuels (the FI stands for “fossil fuel intensive”). That’s basically the world preferred by the oil industry and the US Republican party, i.e. “Drill, baby, drill”.
• In contrast, A1B represents similar economic trends, but more of a balance of energy sources – i.e. something closer to what Obama was advocating in his state-of-the union address.
• A2 is an intermediate scenario, with less globalization, and less technological development, and slower growth in the developing world.
• B1 is what we might get if the world gets its act together and agrees tough new targets to reduce global emissions, and then actually follows through and implements them – i.e. something dramatically different to the Kyoto experience.

The data comes directly from the science (i.e. from the models for future projections, and from observations for the years prior to 2000). But the choices of how to present the information are not scientific choices, they are value choices. The choices made in the first graph all tend to play down the seriousness of climate change, while the choices in the second graph all tend to emphasize it. In particular, the choice not to include A1FI, the business-as-usual path in the first graph could be argued as a very serious omission – a failure to warn the world how bad it could get on our current path. Similarly, the decision to extend only the lower scenarios into future centuries conveys an overall message that we get to choose between two paths, one that stabilizes around +2°C and one that stabilizes around the +3°C level. This is not a fair representation of today’s policy choices.

Okay, now I should say where I got the two graphs from. The first might be very familiar – it’s from the IPCC 2007 assessment. The second is from the Copenhagen Diagnosis in 2009, a document put together by a respectable group of scientists (many of them are IPCC lead authors), intended as an update on the last IPCC report, taking on board developments in the science, and in particular, a growing body of evidence that the IPCC projections have tended to underestimate the trends.

The question of which graph better represents the prognosis is clearly a value judgment. Having compared the two, I now feel that the IPCC graph is missing a major part of the story, and hence is misleading. I think there are weaknesses in the second graph too, as the compressed timescale for the 21st century makes it really hard to discern the three trends. But it certainly seems a lot more appropriate to include more of the pre-industrial context, and to choose a pre-industrial temperature baseline. These graphs have the potential to take on an iconic status, and to directly affect people’s thinking about climate change. We really ought to examine more closely the choices that were made in presenting them.

Update, Feb 3 2011: Bart does a similar comparison with a third graph, which does a better job of the pre-industrial reconstruction, but still suffers from all the other problems of the IPCC graph.

I spent some time this week explaining to my undergraduate class the ideas of thermal equilibrium (loosely speaking, the point at which the planet’s incoming solar radiation and outgoing blackbody radiation are in balance) and climate sensitivity (loosely speaking, how much warmer the earth will get per doubling of CO2, until it reaches a new equilibrium). I think some of my students might prefer me to skip the basic physics, and get on quicker to the tough questions of what solutions there are to climate change, whether geo-engineering will work, and the likely impacts around the world.

So it’s nice to be reminded that a good grasp of the basic science is important. A study produced by the Argentinean group Federacion Ecologia Universal, and published on the American Association for Advancement of Science website, looked at the likely impact of climate change on global food supplies by the year 2020, concluding that global food prices will rise by iup to 20%, and some countries, such as India, will see crop yields drop as much as 30%. The study claims to have used IPCC temperature projections of up to 2.4°C rise in global average temperatures by 2020 on a business-as-usual scenario.

The trouble is the IPCC doesn’t have temperature projections anywhere near this high for 2020. As Scott Mandia explains, it looks like the author of the report made a critical (but understandable) mistake, confusing the two ways of understanding ‘climate sensitivity’:

• equilibrium climate sensitivity, which means overall eventual global temperature rise that would result  if we double the level of CO2 concentrations in the atmosphere.
• transient climate sensitivity response, which means the actual temperature rise the planet will have experienced at the time this doubling happens.

These are different quantities because of lags in the system. It takes many years (perhaps decades) for the earth to reach a new equilibrium whenever we increase the concentrations of greenhouse gases, because most of the extra energy is initially absorbed by the oceans, and it takes a long time for the oceans and atmosphere to settle into a new balance. By global temperature, scientists normally mean the average air temperature measured just above the surface (which is probably where temperature matters most to humans).

BTW, calculating the temperature rise “per doubling of CO2″ make sense because the greenhouse effect is roughly logarithmic – each doubling produces about the same temperature rise. So for example, the pre-industrial concentration of CO2 was about 280ppm (parts per million). So a doubling would take us to 560ppm (we’re currently at 390ppm).

To estimate how quickly the earth will warm, and where the heat might go, we need good models of how the earth systems (ocean, atmosphere, ice sheets, land surfaces) move heat around. In earth system models, the two temperature responses are estimated from two different types of experiment:

• equilibrium climate sensitivity is calculated by letting CO2 concentrations rise steadily over a number of years, until they reach double the pre-industrial levels. They are then held steady after this point, and the run continues until the global temperature stops changing.
• transient climate sensitivity response is calculated by increasing CO2 concentrations by 1% per year, until they reach double the pre-industrial levels, and taking the average temperature at that point.

Both experiments are somewhat unrealistic, and should be thought of more as thought experiments rather than predictions. For example, in the equilibrium experiment, it’s unlikely that CO2 concentrations would stop rising and then remain constant from that point on. In the transient experiment, the annual rise of 1% is now unrealistic –  CO2 concentrations rose by more than 2% per year over the last decade. So, the IPCC figures for transient sensitivityresponse are probably higher than what we’ll actually experience when we do reach 560ppm, because we’ll be getting there quicker. On the other hand, the IPCC figures for equilibrium sensitivity are probably lower than what we’ll eventually experience if we do reach 560ppm, because if we reach that level, we then probably won’t be able to prevent CO2 concentrations going even higher.

Understanding all this matters for many reasons. If people confuse the two types of sensitivity, they’ll misunderstand what temperature changes are likely to happen when. More importantly, failure to understand these ideas means a failure to understand the lags in the system:

• there’s a lag of decades between increasing greenhouse gas concentrations and the eventual temperature response. In other words, we’re always owed more warming than we’ve had. Even if we stopped using fossil fuels immediately, temperatures would still rise for a while.
• there’s another lag also decades long, between peak emissions and peak concentrations. If we get greenhouse gas emissions under control and then start to reduce them, atmospheric concentrations will continue to rise for as long as the emissions exceed the rate of natural removal of CO2 from the atmosphere.
• there’s another lag (and evidence shows it’s also decades long) between humans realising climate change is a serious problem, and any coordinated attempts to do something about it.
• and yet another lag (probably also decades long, hopefully shorter) between the time we implement any serious international climate policies and the point at which we reach peak emissions, because it will take a long time to re-engineer the world’s energy infrastructure to run on non-fossil fuel energy.

Add up these lags, and it becomes apparent that climate change is a problem that will stretch most people’s imaginations. We’re not used to having to having to plan decades ahead, and we’re not used to the idea that any solution will take decades before it starts to make a difference.

And of course, if people who lie about climate change for a living merely say “ha, ha, a scientist made a mistake so global warming must be a myth!” we’ll never get anywhere. Indeed, we may even have already caused the impacts on food supply described in the withdrawn report. It’s just that it’s likely to take longer than 2020 before we see them played out.

Last week I attended the workshop in Exeter to lay out the groundwork for building a new surface temperature record. My head is still buzzing with all the ideas we kicked around, and it was a steep learning curve for me because I wasn’t familiar with many of the details (and difficulties) of research in this area. In many ways it epitomizes what Paul Edwards terms “Data Friction” – the sheer complexity of moving data around in the global observing system means there are many points where it needs to be transformed from one form to another, each of which requires people’s energy and time, and, just like real friction, generates waste and slows down the system. (Oh, and some of these data transformations seem to generate a lot of heat too, which rather excites the atoms of the blogosphere).

Which brings us to the reasons the workshop existed in the first place. In many ways, it’s a necessary reaction to the media frenzy over the last year or so around alleged scandals in climate science, in which scientists are supposed to be hiding or fabricating data, which has allowed the ignoranti to pretend that the whole of climate science is discredited. However, while the nature and pace of the surface temperatures initiative has clearly been given a shot in the arm by this media frenzy, the roots of the workshop go back several years, and have a strong scientific foundation. Quite simply, scientists have recognized for years that we need a more complete and consistent surface temperature record with a much higher temporal resolution than currently exists. Current long term climatological records are mainly based on monthly summary data. Which is inadequate to meet the needs of current climate assessment, particularly the need for better understanding of the impact of climate change on extreme weather. Most weather extremes don’t show up in the monthly data, because they are shorter term – lasting for a few days or even just a few hours. This is not always true of course; Albert Klein Tank pointed out in his talk that this summer’s heatwave in Moscow occured mainly in a single calendar month, and hence shows up strongly in the monthly record. But in general, that is unusual, and so the worry is that monthly records tend to mask the occurrence of extremes (and hence may conceal trends in extremes).

The opening talks at the workshop also pointed out that the intense public scrutiny puts us in a whole new world, and one that many of the workshop attendees are clearly still struggling to come to terms with. Now, it’s clear that any new temperature record needs to be entirely open and transparent, so that every piece of research based on it could (in principle) be traced all the way back to basic observational records, and to echo the way John Christy put it at the workshop – every step of the research now has to be available as admissible evidence that could stand up in a court of law, because that’s the kind of scrutiny we’re being subjected to. Of course, the problem is that not only isn’t science ready for this (no field of science is anywhere near that transparent), it’s also not currently feasible, given the huge array of data sources being drawn on, the complexities of ownership and access rights, the expectations that much of the data will have high commercial value.

I’ll attempt a summary, but it will be rather long, as I don’t have time to make it any shorter. The slides from the workshop are now all available, and the outcomes from the workshop will be posted soon. The main goals were summarized in Peter Thorne’s opening talk: to create a (longish) list of principles, a roadmap for how to proceed, an identification of any overlapping initiatives so that synergies can be exploited, an agree method to engage with broader audiences (including the general public), and an initial governance model.

Did we achieve that? Well, you can skip to the end and see the summary slides, and judge for yourself. Personally, I thought the results were mixed. One obvious problem is that there is no funding on the table for this initiative, and it’s being launched at a time when everyone is cutting budgets, especially in the UK. Which meant that occasionally it felt like we were putting together a Heath Robinson device (Rube Goldberg to you Americans) – cobbling it together out of whatever we could find lying around. Which is ironic really given that the major international bodies (e.g. WMO) seem to fully appreciate the importance of this. And of course, the fact that it will be a vital part of our ability to assess the impacts of climate change over the next few decades.

Another problem is that the workshop attendees struggled to reach consensus on some of the most important principles. For example, should the databank be entirely open, or does it need a restricted section? The argument for the latter is that large parts of the source data are not currently open, as the various national weather services that collect it charge a fee on a cost recovery basis, and wish to restrict access to non-commercial uses as commercial applications are (in some cases) a significant portion of their operating budgets. The problem is that while the monthly data has been shared freely with international partners for many years, the daily and sub-daily records have not, because these are the basis for commercial weather forecasting services. So an insistence on full openness might mean a very incomplete dataset, which then defeats the purpose, as researchers will continue to use other (private) sources for more complete records.

And what about an appropriate licensing model? Some people argued that the data must be restricted to non-commercial uses, because that’s likely to make negotiations with national weather services easier. But others argued that unrestricted licenses should be used, so that the databank can help to lay the foundation for the development of a climate services industry (which would create jobs, and therefore please governments). [Personally, I felt that if governments really want to foster the creation of such an industry, then they ought to show more willingness to invest in this initiative, and until they do, we shouldn't pander to them. I'd go for a cc by-nc-sa license myself, but I think I was outvoted]. Again, existing agreements are likely to get in the way: 70% of the European data would not be available if the research-only clause clause was removed.

There was also some serious disagreement about timelines. Peter outlined a cautious roadmap that focussed on building momentum, and delivering the occasional reports and white papers over the next year or so. The few industrial folks in the audience (most notably, Amy Luers from Google) nearly choked on their cookies – they’d be rolling out a beta version of the software within a couple of weeks if they were running the project. Quite clearly, as Amy urged in her talk, the project needs to plan for software needs right from the start, release early, prepare for iteration and flexibility, and invest in good visualizations.

Oh, and there wasn’t much agreement on open source software either. The more software oriented participants (most notably, Nick Barnes, from the Climate Code Foundation) argued strongly that all software, including every tool used to process the data every step of the way should be available as open source. But for many of the scientists, this represented a huge culture change. There was even some confusion about what open source means (e.g. that ‘open’ and ‘free’ aren’t necessarily the same thing).

On the other hand, some great progress was made in many areas, including identifying many important data services, building on lessons learnt from other large climate and weather data curation efforts, offers of help from many of the international partners (including offers of data from NCDC, NCAR, EURO4M, from across Europe and North America, as well as Russia, China, Indonesia, and Argentina). Agreement was clear that version control and good metadata are vital, and need to be planned for right from the start, but also that providing full provenance for each data item is an important long term goal, but cannot be a rule from the start, as we will have to build on existing data sources that come with little or no provenance information. Oh, and I was very impressed with the deep thinking and planning around benchmarking for homogenization tools (I’ll blog more on this soon, as it fascinates me).

Oh, and on the size of the task. Estimates of the number of undigitized paper records in the basements of various weather services ran to hundreds of millions of pages. But I still didn’t get a sense of the overall size of the planned databank…

Things I learnt:

• Steve Worley from NCAR, reflecting on lessons from running ICOADS, pointed out that no matter how careful you think you’ve been, people will end up mis-using the data because they ignore or don’t understand the flags in the metadata.
• Steve also pointed out that a drawback with open datasets is the proliferation of secondary archives, which then tend to get out of date and mislead users (as they rarely direct users back to the authoritative source).
• Oh, and the scope of the uses of such data is usually surprisingly large and diverse.
• Jay Lawrimore, reflecting on lessons from NCDC, pointed out that monthly data and daily and sub-daily data are collected and curated along independent routes, which then makes it hard to reconcile them. The station names sometimes don’t match, the lat/long coords don’t match (e.g. because of differences in rounding), and the summarized data are similar but not exact.
• Another problem is that it’s not always clear exactly which 24-hour period a daily summary refers to (e.g. did they use a local or UTC midnight?). Oh, and this also means that 3- and 6-hour synoptic readings might not match the daily summaries either.
• Some data doesn’t get transmitted, and so has to be obtained later, even to the point of having to re-key it from emails. Long delays in obtaining some of the data mean the datasets frequently have to be re-released.
• Personal contacts and workshops in different parts of the world play a surprisingly important role in tracking down some of the harder to obtain data.
• NCDC runs a service called Datzilla (similar to Bugzilla for software) for recording and tracking reported defects in the dataset.
• Albert Klein Tank, describing the challenges in regional assessment of climate change and extremes, pointed out that the data requirements for analyzing extreme events are much higher than for assessing global temperature change. For example, we might need to know not just how many days were above 25°C compared to normal, but also how much did it cool off overnight (because heat stress and human health depend much more on overnight relief from the heat).
• John Christy, introducing the breakout group on data provenance, had some nice examples in his slides of the kinds of paper records they have to deal with, and a fascinating example of a surface station that’s now under a lake, and hence old maps are needed to pinpoint its location.
• From Michael de Podesta, who insisted on a healthy dose of serious metrology (not to be confused with meteorology): All measurements ought to come with an estimation of uncertainty, and people usually make a mess of this because they confuse accuracy and precision.
• Uncertainty information isn’t metadata, it’s data. [Oh, and for that matter anything that's metadata to one community is likely to be data to another. But that's probably confusing things too much]
• Oh, and of course, we have to distinguish Type A and Type B uncertainty. Type A is where the uncertainty is describable using statistics, so that collecting bigger samples will reduce it. Type B is where you just don’t know, so that collecting more data cannot reduce the uncertainty.
• From Matt Menne, reflecting on lessons from the GHCN dataset, explaining the need for homogenization (which is climatology jargon for getting rid of errors in the observational data that arise because of changes over time in the way the data was measured). Some of the inhomogeneities are due to abrupt changes (e.g. because a recording station was moved, or got a new instrument), and also gradual changes (e.g. because the environment for a recording station slowly changes, e.g. gradual urbanization of its location).
• Matt has lots of interesting examples of inhomogeneities in his slides, includes some really nasty ones. For example, a station in Reno, Nevada, that was originally in town, and then moved to the airport. There’s a gradual upwards trend in the early part of the record, from an urban heat island effect, and another similar trend in the latter part, after it moved to the airport, as the airport was also eventually encroached by urbanisation. But if you correct for both of these, as well as the step change when the station moved, you’re probably over-correcting….
• which led Matt to suggest the Climate Scientist’s version of the Hippocratic Oath: First, do not flag good data as bad; Then do not make bias adjustments where none are warranted.
• While criticism from non-standard sources (that’s polite-speak for crazy denialists) is coming faster than any small group can respond to (that’s code for the CRU), useful allies are beginning to emerge, also from the blogosphere, in the form of serious citizen scientists (such as Zeke Hausfather) who do their own careful reconstructions, and help address some of the crazier accusations from denialists. So there’s an important role in building community with such contributors.
• John Kennedy, talking about homogenization for Sea Surface Temperatures, pointed out that Sea Surface and Land Surface data are entirely different beasts, requiring totally different approaches to homogenization. Why? because SSTs are collected from buckets on ships, engine intakes on ships, drifting buoys, fixed buoys, and so on. Which means you don’t have long series of observations from a fixed site like you do with land data – every observation might be from a different location!

Things I hope I managed to inject into the discussion:

• “solicitation of input from the community at large” is entirely the wrong set of terms for white paper #14. It should be about community building and engagement. It’s never a one-way communication process.
• Part of the community building should be the support for a shared set of open source software tools for analysis and visualization, contributed by the various users of the data. The aim would be for people to share their tools, and help build on what’s in the collection, rather than having everyone re-invent their own software tools. This could be as big a service to the research community as the data itself.
• We desperately need a clear set of use cases for the planned data service (e.g. who wants access to which data product, and what other information will they be needing and why?). Such use cases should illustrate what kinds of transparency and traceability will be needed by users.
• Nobody seems to understand just how much user support will need to be supplied (I think it will be easy for whatever resources are put into this to be overwhelmed, given the scrutiny that temperature records are subjected to these days)…
• The rate of change in this dataset is likely to be much higher than has been seen in past data curation efforts, given the diversity of sources, and the difficulty of recovering complete data records.
• Nobody (other than Bryan) seemed to understand that version control will need to be done at a much finer level of granularity than whole datasets, and that really every single data item needs to have a unique label so that it can be referred to in bug reports, updates, etc. Oh and that the version management plan should allow for major and minor releases, given how often even the lowest data products will change, as more data and provenance information is gradually recovered.
• And of course, the change process itself will be subjected to ridiculous levels of public scrutiny, so the rational for accepting/rejecting changes and scheduling new releases needs to be clear and transparent. Which means far more attention to procedures and formal change control boards than past efforts have used.
• I had lots of suggestions about how to manage the benchmarking effort, including planning for the full lifecycle: making sure the creation of the benchmark is a really community consensus building effort, and planning for retirement of each benchmark, to avoid the problems of overfitting. Susan Sim wrote an entire PhD on this.
• I think the databank will need to come with a regularly updated blog, to provide news about what’s happening with the data releases, highlight examples of how it’s being used, explain interesting anomalies, interpret published papers based on the data, etc. A bit like RealClimate. Oh, and with serious moderation of the comment threads to weed out the crazies. Which implies some serious effort is needed.
• …and I almost but not quite entirely learned how to pronounce the word ‘inhomogeneities’ without tripping over my tongue. I’m just going to call them ‘bugs’.

Update Sept 21, 2010: Some other reports from the workshop.

Here’s an appalling article by Andy Revkin on dotEarth which epitomizes everything that is wrong with media coverage of climate change. Far from using his position to educate and influence the public by seeking the truth, journalists like Revkin now seem to have taken to just making shit up, reporting what he reads in blogs as the truth, rather than investigating for himself what scientists actually do.

Revkin kicks off by citing a Harvard cognitive scientist found guilty of academic misconduct, and connecting it with “assertions that climate research suffered far too much from group think, protective tribalism and willingness to spin findings to suit an environmental agenda”. Note the juxtaposition. On the one hand, a story of a lone scientist who turned out to be corrupt (which is rare, but does happen from time to time). On the other hand, a set of insinuations about thousands of climate scientists, with no evidence whatsoever. Groupthink? Tribalism? Spin? Can Revkin substantiate these allegations? Does he even try? Of course not. He just repeats a lot of gossip from a bunch of politically motivated blogs, and demonstrates his own total ignorance of how scientists work.

He does offer two pieces of evidence to back up his assertion of bias. The first is the well-publicized mistake in the IPCC report on the retreat of the Himalayan glaciers. Unfortunately, the quotes from the IPCC authors in the very article Revkin points to, show it was the result of an honest mistake, despite an entire cadre of journalists and bloggers trying to spin it into some vast conspiracy theory. The second is about a paper on the connection between vanishing frogs and climate change, cited in the IPCC report. The IPCC report quite correctly cites the paper, and gives a one sentence summary of it. Somehow or other, Revkin seems to think this is bias or spin. It must have entirely escaped his notice that the IPCC report is supposed to summarize the literature in order to assess our current understanding of the science. Some of that literature is tentative, and some less so. Now, maybe Revkin has evidence that there is absolutely no connection between the vanishing frogs and climate change. If so, he completely fails to mention it. Which means that the IPCC is merely reporting on the best information we have on the subject. Come on Andy, if you want to demonstrate a pattern of bias in the IPCC reports, you’re gonna have to work damn harder than that. Oh, but I forgot. You’re just repeating a bunch of conspiracy theories to pretend you have something useful to say, rather than actually, say, investigating a story.

From here, Revkin weaves a picture of climate science as “done by very small tribes (sea ice folks, glacier folks, modelers, climate-ecologists, etc)”, and hence suggests they must therefore be guilty of groupthink and confirmation bias. Does he offer any evidence for this tribalism? No he does not, for there is none. He merely repeats the allegations of a bunch of people like Steve McIntyre, who working on the fringes of science, clearly do belong to a minor tribe, one that does not interact in any meaningful way with real climate scientists. So, I guess we’re meant to conclude that because McIntyre and a few others have formed a little insular tribe, that this must mean mainstream climate scientists are tribal too? Such reasoning would be laughable, if this wasn’t such a serious subject.

Revkin claims to have been “following the global warming saga – science and policy – for nearly a quarter century”. Unfortunately, in all that time, he doesn’t appear to have actually educated himself about how the science is done. If he’d spent any time in a climate science research institute, he’d know this allegation of tribalism is about as far from the truth as it’s possible to get. Oh, but of course, actually going and observing scientists in action would require some effort. That seems to be just a little too much to ask.

So, to educate Andy, and to save him the trouble of finding out for himself, let me explain. First, a little bit of history. The modern concern about the potential impacts of climate change probably dates back to the 1957 Revelle and Suess paper, in which they reported that the oceans absorb far less anthropogenic carbon emissions than was previously thought. Revelle was trained in geology and oceanography. Suess was a nuclear physicist, who studied the distribution of carbon-14 in the atmosphere. Their collaboration was inspired by discussions with Libby, a physical chemist famous for the development of radio-carbon dating. As head of the Scripps Institute, Revelle brought together oceanographers with atmospheric physicists (including initiating the Mauna Loa of the measurement of carbon dioxide concentrations in the atmosphere), atomic physicists studying dispersal of radioactive particles, and biologists studying the biological impacts of  radiation. Tribalism? How about some truly remarkable inter-disciplinary research?

I suppose Revkin might argue that those were the old days, and maybe things have gone downhill since then. But again, the evidence says otherwise. In the 1970′s, the idea of earth system science began to emerge, and in the last decade, it has become central to the efforts to build climate simulation models to improve our understandings of the connections between the various earth subsystems: atmosphere, ocean, atmospheric chemistry, ocean biogeochemistry, biology, hydrology, glaciology and meteorology. If you visit any of the major climate research labs today, you’ll find a collection of scientists from many of these different disciplines working alongside one another, collaborating on the development of integrated models, and discussing the connections between the different earth subsystems. For example, when I visited the UK Met Office two years ago, I was struck by their use of cross-disciplinary teams to investigate specific problems in the simulation models. When I visited, they had just formed such a cross-disciplinary team to investigate how to improve the simulation of the Indian monsoons in their earth system models. This week, I’m just wrapping up a month long visit to the Max Planck Institute for Meteorology in Hamburg, where I’ve also regularly sat in on meetings between scientists from the various disciplines, sharing ideas about, for example, the relationships between atmospheric radiative transfer and ocean plankton models.

The folks in Hamburg have been kind enough to allow me to sit in on their summer school this week, in which they’re training the next generation of earth science PhD students how to work with earth system models. The students are from a wide variety of disciplines: some study glaciers, some clouds, some oceanography, some biology, and so on. The set of experiments we’ve been given to try out the model include: changing the cloud top mass flux, altering the rate of decomposition in soils, changing the ocean mixing ratio, altering the ocean albedo, and changing the shape of the earth. Oh, and they’ve mixed up the students, so they have to work in pairs with people from another discipline. Tribalism? No, right from the get go, PhD training includes the encouragement of cross-disciplinary thinking and cross-disciplinary working.

Of course, if Revkin ever did wander into a climate science research institute he would see this for himself. But no, he prefers pontificating from the comfort of his armchair, repeating nonsense allegations he reads on the internet. And this is the standard that journalists hold for themselves? No wonder the general public is confused about climate change. Instead of trying to pick holes in a science they clearly don’t understand, maybe people like Revkin ought to do some soul searching and investigate the gaping holes in journalistic coverage of climate change. Then finally we might find out where the real biases lie.

So, here’s a challenge for Andy Revkin: Do not write another word about climate science until you have spent one whole month as a visitor in a climate research institute. Attend the seminars, talk to the PhD students, sit in on meetings, find out what actually goes on in these places. If you can’t be bothered to do that, then please STFU [about this whole bias, groupthink and tribalism meme].

Update: On reflection, I think I was too generous to Revkin when I accused him of making stuff up, so I deleted that bit. He’s really just parroting other people who make stuff up.

Update #2: Oh, did I mention that I’m a computer scientist? I’ve been welcomed into various climate research labs, invited to sit in on meetings and observe their working practices, and to spend my time hanging out with all sorts of scientists from all sorts of disciplines. Because obviously they’re a bunch of tribalists who are trying to hide what they do. NOT.

Update #3: I’ve added a clarifying rider to my last paragraph  - I don’t mean to suggest Andy should shut up altogether, just specifically about these ridiculous memes about tribalism and so on.

The Muir Russell report came out today, and I just finished reading the thing. It should be no surprise to anyone paying attention that it completely demolishes the the allegations that have been made about the supposed bad behaviour of the CRU research team. But overall, I’m extremely disappointed, because the report completely misses the wood for the trees. It devotes over 100 pages to a painstaking walk through every single allegation made against the CRU, assessing the evidence for each, and demolishing them one after another. The worst it can find to say about the CRU is that it hasn’t been out there in the lead over the last decade in responding to the new FoI laws, adapting to the rise of the blogosphere, and adapting to changing culture of openness for scientific data. The report makes a number of recommendations for improvements in processes and practices at the CRU, and so can be taken as mildly critical, especially of CRU governance. But in so doing, it never really acknowledges the problems a small research unit (varying between 3.5 to 5 FTE staff over the last decade) would have in finding the resources and funding to be an early adopter in open data and public communication, while somehow managing to do cutting edge research in its area of expertise too. Sheesh!

But my biggest beef with the report is that nowhere, in 100 pages of report plus 60 pages of appendices, does it ever piece together the pattern represented by the set of allegations it investigates. Which means it achieves nothing more than being one more exoneration in a very long list of exonerations of climate scientists. It will do nothing to stop the flood of hostile attacks on science, because it never once considers the nature of those attacks. Let’s survey some of the missed opportunities…

I’m pleased to see the report cite some of the research literature on the nature of electronic communication (e.g. the early work of Sara Kiesler et al), but it’s a really pity they didn’t read much of this literature. One problem recognized even in early studies of email communication is the requesters/informers imbalance. Electronic communication makes it much easier for large numbers of people to offload information retrieval tasks onto others, and receivers of such requests find it hard to figure out which requests they are obliged to respond to. They end up being swamped. Which is exactly what happened with that (tiny) research unit in the UK, when a bunch of self-styled auditors went after them.

And similar imbalances pervade everything. For example on p42, we have:

“There continues to be a scientific debate about the reality, causes and uncertainties of climate change that is conducted through the conventional mechanisms of peer-reviewed publication of results, but this has been paralleled by a more vociferous, more polarised debate in the blogosphere and in popular books. In this the protagonists tend to be divided between those who believe that climate is changing and that human activities are contributing strongly to it, and those that are sceptical of this view. This strand of debate has been more passionate, more rhetorical, highly political and one in which each side frequently doubts the motives and impugns the honesty of the other, a conflict that has fuelled many of the views expressed in the released CRU emails, and one that has also been dramatically fuelled by them.” (page 42, para 26)

But the imbalance is clear. This highly rhetorical debate in the blogosphere occurs between, on the one hand, a group of climate scientists with many years training, and whose expertise is considerable (and the report makes a good job of defending their expertise), and on the other hand, a bunch of amateurs, most of whom have no understanding of how science works, and who are unable to distinguish scientific arguments from ideology. And the failure to recognise this imbalance leads the report to conclude that a suitable remedy is to :

“…urge all scientists to learn to communicate their work in ways that the public can access and understand; and to be open in providing the information that will enable the debate, wherever it occurs, to be conducted objectively.” (page 42, para 28)

No, no, no. As I said very strongly earlier this year, this is naive and irresponsible. No scientist can be an effective communicator in a world where people with vested interests will do everything they can to destroy his or her reputation.

Chapter 6 of the report, on the land station temperature record ought to shut Steve McKitrick McIntyre up forever. But of course it won’t, because he’s not interested in truth, only in the dogged determination to find fault with climate scientists’ work no matter what. Here’s some beautiful quotes:

“To carry out the analysis we obtained raw primary instrumental temperature station data. This can be obtained either directly from the appropriate National Meteorological Office (NMO) or by consulting the World Weather Records (WWR) …[web links elided] … Anyone working in this area would have knowledge of the availability of data from these sources.” (Page 46, paras 13-14)

“Any independent researcher may freely obtain the primary station data. It is impossible for a third party to withhold access to the data.” (Page 48, para 20).

…well, anyone that it except McKitrickMcIntyre and followers, who continue to insist, despite all evidence to the contrary, that climate scientists are withholding station data.

And on sharing the code, the report is equally dismissive of the allegations:

“The computer code required to read and analyse the instrumental temperature data is straightforward to write based upon the published literature.  It amounts a few hundred lines of executable code (i.e. ignoring spaces and comments). Such code could be written by any research unit which is competent to reproduce or test the CRUTEM analysis.  For the trial analysis of the Review Team, the code was written in less than two days and produced results similar to other independent analyses. No information was required from CRU to do this.” (Page 51, para 33)

I like the “any research unit which is competent to reproduce or test the CRUTEM analysis” bit. A lovely British way of saying that  the people making allegations about lack of openness are incompetent. And here’s another wonderful British understatement, referring to ongoing criticism of Briffa’s 1992 work:

“We find it unreasonable that this issue, pertaining to a publication in 1992, should continue to be misrepresented widely to imply some sort of wrongdoing or sloppy science.” (page 62, para 32)

Unreasonable? Unreasonable? It’s an outrage, an outrage I tell you!! (translation provided for those who don’t speak British English).

And there’s that failure to address the imbalance again. In examining the allegations from Boehmer-Christiansen, editor of the notoriously low-quality journal Energy and Environment, that the CRU researchers tried to interfer with the peer-review process, we get the following bits of evidence: An email sent by Boehmer-Christiansen to a variety of people with the subject line “Please take note of potetially [sic] serious scientific fraud by CRU and Met Office.“, and Jones’ eventual reply to her head of department: “I don‟t think there is anything more you can do. I have vented my frustration and have had a considered reply from you“, which leads to the finding:

“We see nothing in these exchanges or in Boehmer-Christiansen’s evidence that supports any allegation that CRU has directly and improperly attempted to influence the journal that she edits. Jones’ response to her accusation of scientific fraud was appropriate, measured and restrained.” (page 66, para 14).

Again, a missed opportunity to comment on the imbalance here. Boehmer-Christiansen is able to make wild and completely unfounded accusations of fraud, and nobody investigates her, while Jones’ reactions to the allegations are endlessly dissected, and in the end everything’s okay, because his response was “appropriate, measured and restained”. No, that doesn’t make it okay. It means someone failed to ask some serious questions how and why people like Boehmer-Christiansen can be allowed to get away with continual smearing of respected climate scientists.

So, an entire 160 pages, in which the imbalance is never once questioned – the imbalance between the behaviour that’s expected of climate scientists, and the crap that the denialists are allowed to get away with. Someone has to put a stop to their nonsense, but unfortunately, Muir Russell ducked the responsibility.

Postscript: my interest in software engineering issues makes me unable to let this one pass without comment. The final few pages of the report criticize the CRU for poor software development standards:

“We found that, in common with many other small units across a range of universities and disciplines, CRU saw software development as a necessary part of a researcher‘s role, but not resourced in any professional sense.  Small pieces of software were written as required, with whatever level of skill the specific researcher happened to possess.  No formal standards were in place for: Software specification and implementation; Code reviews; and Software testing” (page 103, para 30).

I don’t dispute this – it is common across small units, and it ought to be fixed. However, it’s a real shame the report doesn’t address the lack of resources and funding for this. But wait. Scroll back a few pages…

“The computer code required to read and analyse the instrumental temperature data is straightforward to write [...] It amounts a few hundred lines of executable code [...]  For the trial analysis of the Review Team, the code was written in less than two days and produced results similar to other independent analyses.” (page 51, para 33)

Er, several hundred lines of code written in less than 2 days? What, with full software specification, code review, and good quality testing standards? I don’t think so. Ironic that the review team can criticize the CRU software practices, while taking the same approach themselves. Surely they must have spotted the irony?? But, apparently not. The hypocrisy that’s endemic across the software industry strikes again: everyone has strong opinions about what other groups ought to be doing, but nobody practices what they preach.

The IPCC schedule impacts nearly all aspects of climate science. At the start of this week’s CCSM workshop, Thomas Stocker from the University of Bern, and co-chair of working group 1 of the IPCC, gave an overview of the road toward the fifth assessment report (AR5), due to be released in 2013

First, Thomas reminded us that the IPCC does not perform science (it’s job is to assess the current state of the science), but increasingly it stimulates science. This causes some tension though, as curiosity-driven research must remain the priority for the scientific community.

The highly politicized environment also poses a huge risk. There are some groups actively seeking to discredit climate science and damage the IPCC, which means that rigor of the IPCC procedures are now particularly important. One important lesson from the last year is that there is no procedure for correcting serious errors in the assessment reports. Minor errors are routine, and are handled by releasing errata. But this process broke down for bigger issues such as the Himalayan glacier error.

Despite the critics, climate science is about as transparent as a scientific field can be. Anyone can download a climate model and see what’s in there. The IPCC process is founded on four key values (thanks to the advocacy of Susan Solomon): Rigor, Robustness, Transparency, and Comprehensiveness. However, there are clearly practical limits to transparency. For example, it’s not possible to open up lead author meetings, because the scientists need to be able to work together in a constructive atmosphere, rather than “having miscellaneous bloggers in the room”!

The structure of the IPCC remain the same: three working groups: WG1 on the physical science basis, WG2 on impacts and adaptation, and WG3 on mitigation, along with a task force on GHG inventories.

The most important principles for the IPCC are in article 2 and 3:

2. “The role of the IPCC is to assess on a comprehensive, objective, open and transparent basis the scientific, technical and socio-economic information relevant to understanding the scientific basis of risk of human-induced climate change, its potential impacts and options for adaptation and mitigation. IPCC reports should be neutral with respect to policy, although they may need to deal objectively with scientific, technical and socio-economic factors relevant to the application of particular policies.

3. Review is an essential part of the IPCC process. Since the IPCC is an intergovernmental body, review of IPCC documents should involve both peer review by experts and review by governments.

A series of meetings have already occurred in preparation for AR5:

• Mar 2009: An expert meeting on science of alternative greenhouse gas metrics. The met and produced a report.
• Sept 2009: An expert meeting on detection and attribution, which produced a report and a good practice guidance paper [which itself is a great introduction to how attribution studies are done].
• Jan 2010: An expert meeting at NCAR on assessing and combining multi-model projections. The report from this meeting is due in a few weeks, and will also include a good practice guide.
• Jun 2010: A workshop on sea level rise and ice sheet instability, which was needed because of the widespread recognition that AR4 was weak on this issue, perhaps too cautious.
• And in a couple of weeks, in July 2010, a workshop on consistent treatment of uncertainties and risks. This is a cross-Working Group meeting, at which they hope to make progress on getting all three working groups to use the same approach. In the AR4, WG1 developed a standardized language for describing uncertainty, but other working groups have not yet.

Thomas then identified some important emerging questions leading up to AR5.

1. Trends and rates of observed climate change, and in particular, the question of whether climate change has accelerated? Many recent papers and reports indicate that it has; the IPCC needs to figure out how to assess this, especially as there are mixed signals. For example, the decadal trend is accelerating in Arctic sea ice extent, but  the global temperature anomaly has not accelerated over this time period.
2. Stability of the Western and Eastern Antarctic ice sheets (WAIS and EAIS). There has been much more dynamic change at margins of these ice sheets, accelerating mass loss, as observed by GRACE. The assessment needs to look into whether these really are accelerating trends, or if its just an artefact of limited duration of measurements.
3. Irreversibilities and abrupt change: how robust and accurate is our understanding? For example, what long term commitment have been made already in sea level rise. And what about commitments in the hydrological cycle, where some regions (Africa, Europe) might go beyond the range of observed drought within the next couple of decades, and this may be unavoidable.
4. Clouds and Aerosols, which will have their own entire chapter in AR5. There are still big uncertainties here. For example, low level clouds are a positive feedback in the north-east Pacific, yet all but one model are unable to simulate this.
5. Carbon and other biogeochemical cycles. New ice core reconstructions were published just after AR4, and give us more insights into regional carbon cycle footprints caused by abrupt climate change in the past. For example, the ice cores show clear changes in soil moisture and total carbon stored  in the Amazon region.
6. Near-term and long-term projections, for example the question of how reliable the decadal projections are. This is a difficult area. Some people say we already have seamless prediction (from decades to centuries), but not Thomas is not yet convinced. For example, there are alarming new results on number of extreme hot days across southern Europe that need to be assessed – these appear to challenge assumptions about the decadal trends.
7. Regional issues – eg frequency and severity of impacts. Traditionally, the IPCC reports have taken an encyclopedic approach: take each region, and list the impacts in each. Instead, for AR5, the plan is to start with the physical processes, and then say something about sensitivity within each region to these processes.

Here’s an overview of the planned structure of the AR5 WG1 report:

• Intro
• 4 chps on observations and paleoclimate
• 2 chps on process understanding (biogeochemistry and clouds/aerosols)
• 3 chps from forcing to attributions
• 2 chps on future climate change and predictability (near term and long term)
• 2 integration chapters (one on sea level rise, and one on regional issues)

Some changes are evident from AR4. Observations have become more important. They grew to 3 chapters in AR4, and will keep the same in AR5. There will be another crack at paleoclimate, and new chapters on: sea level rise (a serious omission in AR4); clouds and aerosols; the carbon cycle; and regional change. There is also a proposal to produce an atlas which will include a series of maps summarizing the regional issues.

The final draft of the WG1 report is due in May 2013, with a final plenary in Sept 2013. WG2 will finish in March 2014, and WG3 in April 2014. Finally, the IPCC Synthesis Report is to be done no later than 12 months from WG1 report, ie. by September 2014. There has been pressure to create a process that incorporates new science throughout 2014 in to the synthesis report, however Thomas has successfully opposed this, on the basis that it will cause far more controversy if the synthesis report is not consistent with the WG reports.

The deadlines for published research to be included in the assessment is as follows. Papers need to be submitted for publication by 31 July 2012, and must be in press by 15 March 2013. The IPCC has to be very strict about this, because there are people out there who have nothing better to do than to wade through all the references in AR4 and check that all of them appeared before the cutoff date.

Of course, these dates are very relevant to the CCSM workshop audience. Thomas urged everyone not to leave this to the last minute; journal editors and reviewers will be swamped if everyone tries to get their papers published just prior to the deadline [although I suspect this is inevitable?].

Finally, here is a significant challenge in communication coming up. For AR5 we’re expecting to see a much broader model diversity than in previous assessments, partly because there are more models (and more variants), and partly because the models now include a broader range of earth system processes. This will almost certainly mean a bigger model spread,  and hence a likely increase in uncertainty. It will be a significant challenge to communicate the reasons for this to policymakers and a lay audience. Thomas argues that we must not be ashamed to present how science works – that in some cases the uncertainties multiply, during which the spread of projections grows, and then when we get the models more constrained by observations they converge again. But this also poses problems in how we do model elimination and model weighting in ensemble projections. For example, if a particular model shows no sea ice in the year 2000, it probably should be excluded as this is clearly wrong. But how do we set clear criteria for this?

I thought I wouldn’t blog any more about the CRU emails story, but this one is very close to my heart, so I can’t pass it up. Brian Angliss, over at Scholars and Rogues, has written an excellent piece on the lack of context in the stolen emails, and the reliability of any conclusions that might be based on them. To support his analysis, he quotes extensively from the paper “the Secret Life of Bugs” by Jorge Aranda and Gena Venolia from last year’s ICSE, in which they convincingly demonstrated that electronic records of discussions about software bugs are frequently unreliable, and that there is a big difference between the recorded discussions and what you find when you actually track down the participants and ask them directly.

BTW Jorge will be defending his PhD thesis in a couple of weeks, and it’s full of interesting ideas about how software teams develop a shared understanding of the software they develop, and the implications that this has on team organisation. I’ll be mining it for ideas to explore in my own studies of climate modellers later this year…