{"id":2080,"date":"2010-12-14T01:53:30","date_gmt":"2010-12-14T06:53:30","guid":{"rendered":"http:\/\/www.easterbrook.ca\/steve\/?p=2080"},"modified":"2010-12-14T01:53:45","modified_gmt":"2010-12-14T06:53:45","slug":"agu-session-on-software-engineering-for-climate-modeling","status":"publish","type":"post","link":"http:\/\/www.easterbrook.ca\/steve\/2010\/12\/agu-session-on-software-engineering-for-climate-modeling\/","title":{"rendered":"AGU session on Software Engineering for Climate Modeling"},"content":{"rendered":"

Here’s the first of a series of posts from the American Geophysical Society (AGU) Fall meeting<\/a>, which is happening this week in San Francisco. The meeting is huge – they’re expecting 19,000 scientists to attend, making it the largest such meeting in the physical sciences.<\/p>\n

The most interesting session today was a new session for the AGU: \u00a0IN14B “Software Engineering for Climate Modeling”. And I’m not just saying that because it included my talk – all the talks were fascinating. (I’ve posted the slides for my talk, “Do Over or Make Do: Climate Models as a Software Development Challenge<\/a>“).<\/p>\n

After my talk, the next speaker was\u00a0Cecelia DeLuca<\/strong><\/a> of NOAA, with a talk entitled “Emergence of a Common Modeling Architecture for Earth System Science”<\/em>. Cecelia gave a great overview of the Earth System Modelling Framework<\/a>. She began by pointing out that climate models don’t just contain science code – they consist of a number of different kinds of software.\u00a0Lots of the code is infrastructure code, which doesn’t necessarily need to be written by scientists.\u00a0Around ten years ago, a number of projects started up that had the aim of building shared, standards-based infrastructure code. The projects needed to develop the technical and mathematical expertise to build infrastructure code. But the advantages of separating this code development from the science code was clear: the teams building infrastructure code could prioritize best practices, run the nightly testing process, etc, whereas typically the scientists would not do this.<\/p>\n

ESMF provides a common modelling architecture.\u00a0Native model data structures (modules, fields, grids, timekeeping) are wrapped into ESMF standard data structures, which conform to relevant standards (E.g.\u00a0ISO standards, CF standards<\/a>, and the Metafor common information model<\/a>, etc).\u00a0The framework also offers runtime compliance checking (e.g. to check timekeeping behaviour is correct), and automated documentation (e.g. the ability to write out model metadata in an XML standard format).<\/p>\n

Because of these efforts, in the US, earth system \u00a0models are converging on a common architecture. It’s built on standardized component interfaces, and creates a layer of structured information within Earth system codes. The lesson here is that if you\u00a0can take the legacy code, and express it in a standard way, you get tremendous power.<\/p>\n

The next speaker was\u00a0Amy Langenhors<\/strong><\/a>t<\/strong> from GFDL, “Making sense of complexity with the FRE climate modelling workflow system”<\/em>. Amy explained the organisational setup at GFDL: there are approximately\u00a0300 people organized into groups:\u00a06 science based groups groups<\/a>, plus a technical services group, and a modelling services group. The latter consists of 15 people, with one of them acting as a\u00a0liaison for each of the science groups. This group provides the software engineering support for the science teams.<\/p>\n

The Flexible Modeling System (FMS)<\/a> is software framework that provides a coupler and infrastructure support.\u00a0FMS releases happen about once per year; it provides an extensive testing framework that currently includes 209 different model configurations.<\/p>\n

One of the biggest challenges for modelling groups like GFDL is the IPCC cycle. Each providing the model runs for the IPCC assessments involves massive complex data processing, for which a good workflow manager is needed.\u00a0FRE is the workflow manager<\/a> for FMS. Development of FRE was started in 2002 by Amy, at a time when the model services group didn’t yet exist.<\/p>\n

FRE includes version control, configuration management, tools for building executables, control of execution, etc. It also provides facilities for creating\u00a0XML model description files, model configuration (using a\u00a0component-based approach), and integrated model testing (e.g. basic tests, restarts, scaling). It also allows for experiment inheritance, so that it’s possible to set up new model configurations based on variants of previous runs, which is useful for perturbation studies.<\/p>\n

Next up was\u00a0Rob Burns<\/a><\/strong> from NASA GSFC, talking about “Software Engineering Practices in the Development of NASA Unified Weather Research and Forecasting (NU-WRF) Model<\/em>“.\u00a0WRF is a weather forecasting model originally developed at NCAR, but widely used across the NWP community. NU-WRF<\/a> is an attempt\u00a0to unify variants of NCAR WRF and to facilitate better use of WRF.\u00a0NU-WRF is built from versions of NCAR’s WRF, with separate process of folding in enhancements.<\/p>\n

As is common with many modelling efforts, there were challenges arising from multiple science teams, with individual goals, interests and expertise, and scientists don’t consider software engineering as their first priority. At NASA, the\u00a0Sofware Integration and Visualization Office (SIVO) provides Software Engineering support for the scientific modelling teams. SIVO helps to drive, but not to lead the scientific modelling efforts. They help with full software lifecycle management, assisting with all software processes from requirements to release, but with domain experts still making the scientific decisions.\u00a0The code is under full version control, using Subversion, and the software engineering team coordinates the effort to get the codes into version control.<\/p>\n

The experience with NU-WRF shows that this kind of partnership between science teams and a software support team can work well.\u00a0Leadership and active engagement with the science teams is needed. However, involvement of the entire science team for decisions is too slow, so a core team was formed to do this.<\/p>\n

The next speaker was\u00a0Thomas Clune<\/a><\/strong> from NASA GISS, with a talk “Constraints and Opportunities in GCM Model Development<\/em>“. Thomas began with the question:\u00a0How did we end up with the software we have today? From a software quality perspective, we wrote the wrong software. Over the years, improvements in fidelity in the models have driven a disproportionate growth in complexity of implementations.<\/p>\n

One important constraint is that model codes change relatively slowly, in part because of the model validation processes – it’s important to be able to validate each code change individually – they can’t be bundled together. But also because code familiarity is important – the scientists have to understand their code, and if it changes too fast, they lose this familiarity.<\/p>\n

However, the problem now is that software quality is incommensurate with the growing socioeconomic role for our models in understanding climate change. There’s a great quote from\u00a0Ward Cunningham<\/a>: “Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as\u00a0interest on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation,\u00a0object-oriented or otherwise\u2026<\/em>” Examples of this debt in climate models include long procedures, kludges, cut-and-paste duplication, short\/ambiguous names, and inconsistent style.<\/p>\n

The opportunities then are to exploit advances in software engineering from elsewhere to systematically and incrementally improve the software quality of climate models.\u00a0For example:<\/p>\n