28. May 2011 · 4 comments · Categories: psychology

This xkcd cartoon made me laugh out loud, because it beautifully captures my attitude towards sports. The mouse-over comment on xkcd points out that it applies to financial analysis too, and (more directly), Dungeons and Dragons:

But then I got thinking. Huge numbers of people are fascinated by the narratives that these commentators produce. There’s something compelling about declaring allegiance to one of the weighted random number generators (sports teams, stock picks, etc), selecting which of the narratives to believe based on that allegiance, and then hoping for (and perhaps betting on) which numbers it will produce. Sometimes the numbers turn out the way you’d hoped, and sometimes they don’t, but either way people prefer to believe in the narratives, rather than acknowledge the randomness. There’s a thrill to the process, but at the same time, a strong sense that everyone else’s narratives are delusional.

What if that’s how most people view science? What if they think that scientists are no different from sports commentators and financial analysts, sounding knowledgeable, but mostly just making up explanations based on wishful thinking? What if people believe that scientists’ explanations are just as unreliable as those of sports commentators, and that therefore you can pick what to believe based on tribal affiliations, rather than on, say, the weight of evidence?

Certainly, that’s how many non-scientist commentators approach climate science. Each new paper published is like a new game outcome. The IPCC team may have won a lot of games in the past, put it’s very unpopular among some groups of fans. Results that suggest some other teams are on the rise can be spun into fabulous narratives about how the IPCC team is past its prime, and cruising towards a big upset.

For those who have no idea what science does, the caption for the cartoon might just as well read “All climate science”.

I picked up a fascinating book today – “The Handbook of Sustainability Literacy” edited by Arran Stibbe. It’s a set of short essays on each of a long list of skills needed for thinking about and achieving sustainability. The contents listing makes worthwhile reading on it’s own, covering many of the things I’ve been reading up on for the last few months. I wonder if it’s possible to design an education program that fosters all these skills:

  • ECOCRITICISM – the ability to investigate cultural artefacts from an ecological perspective.
  • OPTIMISATION – the art of personal sufficiency.
  • GROUNDED ECONOMIC AWARENESS – economic awareness grounded in ecological and ethical values.
  • ADVERTISING AWARENESS – the ability to expose advertising discourses that undermine sustainability, and resist them.
  • TRANSITION SKILLS – skills for transition to a post fossil-fuel age.
  • COMMONS THINKING – the ability to envisage and enable a viable future through connected action.
  • EFFORTLESS ACTION – the ability to fulfil human needs effortlessly through working with nature.
  • PERMACULTURE DESIGN – designing our lives with nature as the model.
  • COMMUNITY GARDENING – skills for building community and working within environmental limits.
  • ECOLOGICAL INTELLIGENCE – viewing the world relationally.
  • SYSTEMS THINKING – the ability to recognize and analyse the inter-connections within and between systems.
  • GAIA AWARENESS – awareness of the animate qualities of the Earth.
  • FUTURES THINKING – the ability to envision scenarios for the future and work towards bringing desirable ones into being.
  • VALUES REFLECTION AND THE EARTH CHARTER – the ability to critique the values of an unsustainable society and consider alternatives.
  • SOCIAL CONSCIENCE – the ability to reflect on deeply-held opinions about social justice and sustainability.
  • NEW MEDIA LITERACY – communication skills for sustainability.
  • CULTURAL LITERACY – understanding and skills for culturally appropriate communication.
  • CARBON CAPABILITY – understanding, ability and motivation for reducing carbon emissions.
  • GREENING BUSINESS – the ability to drive environmental and sustainability improvements in the workplace.
  • MATERIALS AWARENESS – the ability to expose the hidden impact of materials on sustainability.
  • APPROPRIATE TECHNOLOGY AND APPROPRIATE DESIGN – the ability to design systems, technologies and equipment in an appropriate way.
  • TECHNOLOGY APPRAISAL – the ability to evaluate technological innovations.
  • COMPLEXITY, SYSTEMS THINKING AND PRACTICE – skills for managing complexity.
  • COPING WITH COMPLEXITY – the ability to manage complex sustainability problems.
  • EMOTIONAL WELLBEING – the ability to research and reflect on the roots of emotional wellbeing.
  • FINDING MEANING WITHOUT CONSUMING – the ability to experience meaning, purpose and satisfaction through non-material wealth.
  • BEING-IN-THE-WORLD – the ability to think about the self in interconnection and interdependence with the surrounding world.
  • BEAUTY AS A WAY OF KNOWING – the redemption of knowing through the experience of beauty.

There’s a few things I’m might add (social networking and social justice spring to mind), and I see they’ve added some additional chapters on the website. But phew, this looks like an extremely valuable book.

Bad news today – we just had a major grant proposal turned down. It’s the same old story – they thought the research we were proposing (on decision support tools for sustainability) was excellent, but criticized, among other things, the level of industrial commitment and our commercialization plans. Seems we’re doomed to live in times where funding agencies expect universities to take on the role of industrial R&D. Oh well.

The three external reviews were very strong. Here’s a typical paragraph from the first review:

I found the overall project to be very compelling from a “need”, potential “payoff’, technical and team perspective. The linkage between seemingly disparate technology arenas–which are indeed connected and synergistic–is especially compelling. The team is clearly capable and has a proven track record of success in each of their areas and as leaders of large projects, overall. The linkage to regional and institutional strengths and partners, in both academic and industrial dimensions, is well done and required for success.

Sounds good huh? I’m reading it through, nodding, liking the sound of what this reviewer is saying. The problem is, this is the wrong review. It’s not a review of our proposal. It’s impossible to tell that from this paragraph, but later on, mixed in with a whole bunch more generic praise, are some comments on manufacturing processes, and polymer-based approaches. That’s definitely not us. Yet I’m named at the top of the form as the PI, along with the title of our proposal. So, this review made it all the way through the panel review process, and nobody noticed it was of the wrong proposal, because most of the review was sufficiently generic that it passed muster on a quick skim-read.

It’s not the first time I’ve seen this happen. It happens for paper reviews for journals and conference. It happens for grant proposals. It even happens for tenure and promotion cases (including both of the last two tenure committees I sat on). Since we started using electronic review systems, it happens even more – software errors and human errors seem to conspire to ensure a worrying large proportion of reviews get misfiled.

Which is why every review should start with a one paragraph summary of whatever is being reviewed, in the reviewer’s own words. This acts as a check that the reviewer actually understood what the paper or proposal was about. It allows the journal editor / review panel / promotions committee to immediately spot cases of mis-filed reviews. And it allows the authors, when they receive the reviews, to get the most important feedback of all: how well did they succeed in communicating the main message of the paper/proposal?

Unfortunately, in our case, correcting the mistake is unlikely to change the funding decision (they sunk us on other grounds). But at least I can hope to use it as an example to improve the general standard of reviewing in the future.

Previously I posted on the first two sessions of the workshop on A National Strategy for Advancing Climate Modeling” that was held at NCAR at the end of last month:

  1. What should go into earth system models;
  2. Challenges with hardware, software and human resources;

    The third session focussed on the relationship between models and data.

    Kevin Trenberth kicked off with a talk on Observing Systems. Unfortunately, I missed part of his talk, but I’ll attempt a summary anyway – apologies if it’s incomplete. His main points were that we don’t suffer from a lack of observational data, but from problems with quality, consistency, and characterization of errors. Continuity is a major problem, because much of the observational system was designed for weather forecasting, where consistency of measurement over years and decades isn’t required. Hence, there’s a need for reprocessing and reanalysis of past data, to improve calibration and assess accuracy, and we need benchmarks to measure the effectiveness of reprocessing tools.

    Kevin points out that it’s important to understand that models are used for much more than prediction. They are used:

    • for analysis of observational data, for example to produce global gridded data from the raw observations;
    • to diagnose climate & improve understanding of climate processes (and thence to improve the models);
    • for attribution studies, through experiments to determine climate forcing;
    • for projections and prediction of future climate change;
    • for downscaling to provide regional information about climate impacts;

    Confronting the models with observations is a core activity in earth system modelling. Obviously, it is essential for model evaluation. But observational data is also used to tune the models, for example to remove known systematic biases. Several people at the workshop pointed out that the community needs to do a better job of keeping the data used to tune the models distinct from the data used to evaluate them. For tuning, a number of fields are used – typically top-of-the-atmosphere data such as net shortwave and longwave radiation flux, cloud and clear sky forcing, and cloud fractions. Also, precipitation and surface wind stress, global mean surface temperature, and the period and amplitude of ENSO. Kevin suggests we need to do a better job of collecting information about model tuning from different modelling groups, and ensure model evaluations don’t use the same fields.

    For model evaluation, a number of integrated score metrics have been proposed to summarize correlation, root-mean-squared (rms) error and variance ratios – See for example, Taylor 2001Boer and Lambert 2001Murphy et al, 2004Reichler & Kim 2008.

    But model evaluation and tuning aren’t the only ways in which models and data are brought together. Just as important is re-analysis, where multiple observational datasets are processed through a model to provide more comprehensive (model-like) data products. For this, data assimilation is needed, whereby observational data fields are used to nudge the model at each timestep as it runs.

    Kevin also talked about forward modelling, a technique in which the model used to reproduce the signal that a particular instrument would record, given certain climate conditions. Forward modelling is used for comparing models with ground observations and satellite data. In much of this work, there is an implicit assumption that the satellite data are correct, but in practice, all satellite data have biases, and need re-processing. For this work, the models need good emulation of instrument properties and thresholds. For examples, see: Chepfer, Bony et al, 2010Stubenrauch & Kinne 2009.

    He also talked about some of the problems with existing data and models:

    • nearly all satellite data sets contain large spurious variability associated with changing instruments and satellites, orbital decay/drift, calibration, and changing methods of analysis.
    • simulation of the hydrological cycle is poor, especially in the intertropical convergence zone (ITCZ). Tropical transients are too weak, runoff and recycling is not correct, and the diurnal cycle is poor.
    • there are large differences between datasets for low cloud (see Marchand at al 2010)
    • clouds are not well defined. Partly this is a problem of sensitivity of instruments, compounded by the difficulty of distinguishing between clouds and aerosols.
    • Most models have too much incoming solar radiation in the southern oceans, caused by too few clouds. This makes for warmer oceans and diminished poleward transport, which messes up storm tracking and analysis of ocean transports.

    What is needed to support modelling over the next twenty years? Kevin made the following recommendations:

    • Support observations and development into climate datasets.
    • Support reprocessing and reanalysis.
    • Unify NWP and climate models to exploit short term predictions and confront the models with data.
    • Develop more forward modelling and observation simulators, but with more observational input.
    • Targeted process studies such as GEWEX and analysis of climate extremes, for model evaluation.
    • Target problem areas such as monsoons and tropical precipitation.
    • Carry out a survey of fields used to tune models.
    • Design evaluation and model merit scoring based on fields other than those used in tuning.
    • Promote assessments of observational datasets so modellers know which to use (and not use).
    • Support existing projects, including GSICS, SCOPE-CM, CLARREO, GRUAN,

    Overall, there’s a need for a climate observing system. Process studies should not just be left to the observationists – we need the modellers to get involved.

    The second talk was by Ben Kirtman, on “Predictability, Credibility, and Uncertainty Quantification“. He began by pointing out that there is ongoing debate over what predictability means. Some treat it as an inherent property of the climate system, while others think of it as a model property. Ben distinguished two kinds of predictability:

    • Sensitivity of the climate system to initial conditions (predictability of the first kind);
    • Predictability of the boundary forcing (predictability of the second kind).

    Predictability is enhanced by ensuring specific processes are included. For example, you need to include the MJO if you want to predict ENSO. But model-based estimates of predictability are model dependent. If we want to do a better job of assessing predictability, we have to characterize model uncertainty, and we don’t know how to do this today.

    Good progress has been made on quantifying initial condition uncertainty. We have pretty of good ideas for how to probe this (stochastic optimals, bred vectors, etc.) using ensembles with perturbed initial conditions. But from our understanding of chaos theory (e.g. see the Lorenz attractor), predictability depends on which part of the regime you’re in, so we need to assess the predictability for each particular forecast.

    Uncertainty in external forcing include uncertainties in both the natural and anthropogenic forcing; however this is becoming less of an issue in modelling, as these forcings are better understood. Therefore, the biggest challenge is in quantifying uncertainties in model formulation. These arise because of the discrete representation of climate system, the use of parameterization of subgrid processes, and because of missing processes. Current approaches can be characterized as:

    • a posteriori techniques, such as the multi-model ensembles of opportunity used in IPCC assessments, and perturbed parameters/parameterizations, as used in climateprediction.net.
    • a priori techniques, where we incorporate uncertainty as the model evolves. The idea is that uncertainty is in subscale processes and missing physics. Model this non-locally and stochastically. E.g. backscatter, interactive ensembles to incorporate uncertainty in the coupling.

    The term credibility is even less well defined. Ben asked his students what they understood by the term, and they came up with a simple answer: credibility is the extent to which you use the best available science [which corresponds roughly to my suggestion of what model validation ought to mean]. In the literature, there are a number of other way of expressing credibility:

    • In terms of model bias. For example, Lenny Smith offers a Temporal (or spatial) credibility ratio, calculated as the ratio of the smallest timestep in the model to the smallest duration over which a variable has to be averaged before it compares favourably with observations. This expresses how much averaging over the temporal (or spatial) scale you have to do to make the model look like the data.
    • In terms of whether the ensembles bracket the observations. But the problem here is that you can always pump up an ensemble to do this, and it doesn’t really tell you about probabilistic forecast skill.
    • In terms of model skill. In numerical weather prediction, it’s usual to measure forecast quality using some specific skill metrics.
    • In terms of process fidelity – how well the processes represented in the model capture what is known about those processes in reality. This is a reductionist approach, and depends on the extent to which specific processes can be isolated (both in the model, and in the world).
    • In terms of faith – for example, the modellers’ subjective assessment of how good their model is.

    In the literature, credibility is usually used in a qualitative way to talk about model bias. Hence, in the literature, model bias is roughly synonymous with inverse of credibility. However, in these terms, the models currently have a major credibility gap. For example, Ben showed the annual mean rainfall from a long simulation of CESM1, showing bias with respect to GPCP observations. These show the model struggling to capture the spatial distribution of sea surface temperature (SST), especially in equatorial regions.

    Every climate model has a problem with equatorial sea surface temperatures (SST). A recent paper, Anagnostopoulos et al 2009 makes a big deal of this, and is clearly very hostile to climate modelling. They look at regional biases in temperature and precipitation, where the models are clearly not bracketing observations. I googled the Anagnostopooulos paper while Ben was talking – The first few pages of google hits are dominated by denialist website proclaiming this as a major new study demonstrating the models are poor. It’s amusing that this is treated as news, given that such weaknesses in the models are well known within the modelling community, and discussed in the IPCC report. Meanwhile the hydrologists at the workshop tell me that it’s a third-rate journal, so none of them would pay any attention to this paper.

    Ben argues that these weaknesses need to be removed to increase model credibility. This argument seems a little weak to me. While improving model skill and removing biases are important goals for this community, they don’t necessarily help with model credibility in terms of using the best science (because often replacing an empirically derived parameterization with one that’s more theoretically justified will often reduce model skill). More importantly, those outside the modeling community will have their own definitions of credibility, and they’re unlikely to correspond to those used within the community. Some attention to the ways in which other stakeholders understand model credibility would be useful and interesting.

    In summary, Ben identified a number of important tensions for climate modeling. For example, there are tensions between:

    • the desire to measure prediction skill vs. the desire to explore the limits of predictability;
    • the desire to quantify uncertainty, vs. the push for more resolution and complexity in the models;
    • a priori vs. a posteriori methods of assessing model uncertainty.
    • operational vs. research activities (Many modellers believe the IPCC effort is getting a little out of control – it’s a good exercise, but too demanding on resources);
    • weather vs climate modelling;
    • model diversity vs critical mass;

    Ben urged the community to develop a baseline for climate modelling, capturing best practices for uncertainty estimation.

    During a break in the workshop last week, Cecilia Bitz and I managed to compare notes on our undergrad courses. We’ve both been exploring how to teach ideas about climate modeling to students who are not majoring in earth sciences. Cecilia’s course on Weather and Climate Prediction is a little more advanced than mine, as she had a few math and physics pre-requisites, while mine was open to any first year students. For example, Cecilia managed to get the students to run CAM, and experiment with altering the earth’s orbit. It’s an interesting exercise, as it should lead to plenty of insights into connections between different processes in the earth’s system. One of the challenges is that earth system models aren’t necessarily geared up for this kind of tinkering, so you need good expertise in the model being used to understand which kinds of experiments are likely to make sense. But even so, I’m keen to explore this further, as I think the ability to tinker with such models could be an important tool for promoting a better understanding of how the climate system works, even for younger kids

    Part of what’s missing is elegant user interfaces. EdGCM is better, but still very awkward to use. What I really want is something that’s as intuitive as Angry Birds. Okay, so I’m going to have to compromise somewhere – nonlinear dynamics are a bit more complicated than the flight trajectories of avian slingshot.

    But that’s not all – someone needs to figure out what kinds of experiments students (and school kids) might want to perform, and provide the appropriate control widgets, so they don’t have mess around with code and scripts. Rich Loft tells me there’s a project in the works to do something like this with CESM – I’ll looking forward to that! In the meantime, Rich suggested two examples of simple simulations of dynamical systems that get closer to what I’m looking for:

    • The Shodor Disease model that lets you explore the dynamics of epidemics, with people in separate rooms, where you can adjust how much they can move between rooms, how the disease works, and whether immunization is available. Counter-intuitive lesson: crank up the mortality rate to 100% and (almost) everyone survives!
    • The Shodor Rabbits and Wolves simulation, which lets you explore population dynamics of predators and prey. Counter-intuitive lesson: double the lifespan of the rabbits and they all die out pretty quickly!

    In the last post, I talked about the opening session at the workshop on “A National Strategy for Advancing Climate Modeling”, which focussed on the big picture questions. In the second session, we focussed on the hardware, software and human resources challenges.

    To kick off, Jeremy Kepner from MIT called in via telecon to talk about software issues, from his perspective working on Matlab tools to support computational modeling. He made the point that it’s getting hard to make scientific code work on new architectures, because it’s increasingly hard to find anyone who wants to do the programming. There’s a growing gap between the software stacks used in current web and mobile apps, gaming, and so on, and that used in scientific software. Programmers are used to having new development environments and tools, for example for developing games for Facebook, and regard scientific software development tools as archaic. This means it’s hard to recruit talent from the software world.

    Jeremy quipped that software is an evil thing – the trick is to get people to write as little of it as possible (and he points out that programmers make mistakes at the rate of one per 1,000 lines of code). Hence, we need higher levels of abstraction, with code generated automatically from higher level descriptions. Hence, an important question is whether it’s time to abandon Fortran. He also points out that programmers believe they spend most of their time coding, but in fact, coding is a relatively small part of what they do. At least half of their time is testing, which means that effort to speed up the testing process gives you the most bang for the buck.

    Ricky Rood, Jack Fellows, and Chet Koblinsky then ran a panel on human resources issues. Ricky pointed out that if we are to identify shortages in human resources, we have to be clear about whether we mean for modeling vs. climate science vs. impacts studies vs. users of climate information, and so on. The argument can be made that in terms of absolute numbers there are enough people in the field, but the problems are in getting an appropriate expertise mix / balance, having people at the interfaces between different communities of expertise, a lack of computational people (and not enough emphasis on training our own), and management of fragmented resources.

    Chet pointed out that there’s been a substantial rise in the number of job postings using the term “climate modelling” over the last decade. But there’s still a widespread perception is that there aren’t enough jobs (i.e. more grad students being trained than we have positions for). There are some countervailing voices – for example Pielke argues that universities will always churn out more than enough scientists to support their mission, and there’s a recent BAMs article that explored the question “are we training too many atmospheric scientists?“. The shortage isn’t in the number of people being trained, but in the skills mix.

    We covered a lot of ground in the discussions. I’ll cover just some of the highlights here.

    Several people observed that climate model software development has diverged from mainstream computing. Twenty years ago, academia was the centre of the computing world. Now most computing is in commercial world, and computational scientists have much less leverage than we used to. This means that some tools we rely on might no longer be sustainable. E.g. fortran compilers (and autogenerators?) – fewer users care about these, and so there is less support for transitioning them to new architectures. Climate modeling is a 10+ year endeavour, and we need a long-term basis to maintain continuity.

    Much of the discussion focussed on anticipated disruptive transitions in hardware architectures. Whereas in the past, modellers have relied on faster and faster processors to deliver new computing capacity, this is coming to an end. Advances in clock speed have tailed off, and now its  massive parallelization that delivers the additional computing power. Unfortunately, this means the brute force approach of scaling up current GCM numerical methods on a uniform grid is a dead-end.

    As Bryan Lawrence pointed out, there’s a paradigm change here: computers no longer compute, they produce data. We’re entering an era where CPU time is essentially free, and it’s data wrangling that forms the bottleneck. Massive parallelization of climate models is hard because of the volume of data that must be passed around the system. We can anticipated 1-100 exabyte scale datasets (i.e. this is the size not of the archive, but of the data from a single run of an ensemble). It’s unlikely than any institution will have the ability to evolve their existing codes into this reality.

    The massive parallelization and data volumes also bring another problem. In the past, climate modellers have regarded bit-level reproducibility of climate runs to be crucial, partly because reproducing a run exactly is considered good scientific practice, and partly because it allows many kinds of model test to be automated. The problem is, at the scales we’re talking about, exact bit reproducibility is getting hard to maintain. When we scale up to millions of processors, and terabyte data sets, bit-level failures are frequent enough that exact reproducibility can no longer be guaranteed – if a single bit is corrupted during a model run, it may not matter for the climatology of the run, but it will mean exact reproducibility is impossible. Add to this the fact that in the future, CPUs are likely to be less deterministic, then, as Tim Palmer argued at the AGU meeting, we’ll be forced to fundamentally change our codes, and therefore, maybe we should take the opportunity to make the models probabilistic.

    One recommendation that came out of our discussions is to consider a two track approach for the software. Now that most modeling centres have finished their runs for the current IPCC assessment (AR5), we should plan to evolve current codes towards the next IPCC assessment (AR6), while starting now on developing entirely new software for AR7. The new codes will address i/o issues, new solvers, etc.

    One of the questions the committee posed to the workshop was the potential for hardware-software co-design. The general consensus was that it’s not possible in current funding climate. But even if the funding was available, it’s not clear this is desirable, as the software has much longer useful life than any hardware. Designing for specific hardware instantiations tends to bring major liabilities, and (as my own studies have indicated) there seems to be an inverse correlation between availability of dedicated computing resources and robustness/portability of the software. Things change in climate models all the time, and we need the flexibility to change algorithms, refactor software, etc. This means FPGAs might be a better solution. Dark silicon might push us in this direction anyway.

    Software sharing came up as an important topic, although we didn’t talk about this as much as I would have liked. There seems to be a tendency among modelers to assume that making the code available is sufficient. But as Cecelia Deluca pointed out, from the ESMF experience, community feedback and participation is important. Adoption mandates are not constructive – you want people to adopt software because it works better. One of the big problems here is understandability of shared code. The learning curve is getting bigger, and code sharing between labs is really only possible with a lot of personal interaction. We did speculate that auto-generation of code might help here, because it forces the development of higher level language to describe what’s in a climate model.

    For the human resources question, there was a widespread worry that we don’t have the skills and capacity to deal with anticipated disruptive changes in computational resources. There is a shortage of high quality applicants for model development positions, and many disincentives for people to pursue such a career: the long publication cycle, academic snobbery, and the demands of the IPCC all make model development an unattractive career for grad students and early career scientists. We need a different reward system, so that contributions to the model are rewarded.

    However, it’s also clear that we don’t have enough solid data on this – just lots of anecdotal evidence. We don’t know enough about talent development and capacity to say precisely where the problems are. We identified three distinct roles, which someone amusingly labelled: diagnosticians (who use models and model output in their science), perturbers (who explore new types of runs by making small changes to models) and developers (who do the bulk of model development). Universities produce most of the first, a few of the second, and very few of the third. Furthermore, model developers could be subdivided between people who develop new parameterizations and numerical analysts, although I would add a third category: developers of infrastructure code.

    As well as worrying about training of a new generation of modellers, we also worried about whether the other groups (diagnosticians and perturbers) would have the necessary skillsets. Students are energized by climate change as a societal problem, even if they’re not enticed by a career in earth sciences. Can we capitalize on this, through more interaction with work at the policy/science interface? We also need to make climate modelling more attractive to students, and to connect them more closely with the modeling groups. This could be done through certificate programs for undergrads to bring them into modelling groups, and by bringing grad students into modelling centres in their later grad years. To boost computational skills, we should offer training in earth system science to students in computer science, and expand training for earth system scientists in computational skills.

    Finally, let me end with a few of the suggestions that received a very negative response from many workshop attendees:

    • Should the US be offering only one center’s model to the IPCC for each CMIP round? Currently every major modeling center participates, and many of the centres complain that it dominates their resources during the CMIP exercise. However, participating brings many benefits, including visibility, detailed comparison with other models, and a pressure to improve model quality and model documentation.
    • Should we ditch Fortran and move to a higher level languages? This one didn’t really even get much discussion. My own view is that it’s simply not possible – the community has too much capital tied up in Fortran, and it’s the only language everyone knows.
    • Can we incentivize a mass participation in climate modeling, like the “develop apps for the iphone”? This is an intriguing notion, but one that I don’t think will get much traction, because of the depth of knowledge needed to do anything useful at all in current earth system modeling. Oh, and we’d probably need a different answer to the previous question, too.