Next year’s International Conference on Software Engineering (ICSE), to be held in Zurich, has an interesting conference slogan: Sustainable Software for a Sustainable World
In many ways, ICSE is my community. By that I mean, this is the conference where I have presented my research most often, and is generally my first choice of venue for new papers. This is an important point: one of the most crucial pieces of advice I give to new PhD students is to “find your community”. To be successful as a researcher (and especially as an academic) you have to build a reputation for solid research within an existing research community. Which means figuring out early which community you belong to: who will be the audience for your research results? who will understand your work well enough to review your papers? And eventually, which community will you be looking to for letters of support for job applications, tenure reviews, and so on? And once you’ve figured out which community you belong to, you have to attend the conferences and workshops run by that community, and present your work to them as often as you can, and you have to get to know the senior people in that community. Or rather, they have to get to know you.
The problem is, in recent years, I’ve gone off ICSE. Having spent a lot of time in the last few years mixing with a different research community (climate science, and especially geoscientific model development), I come back to the ICSE community with a different perspective, and what I see now (in general) it is a rather insular community, focussed on a narrow, technical set of research questions that seem largely irrelevant to anything that matters, and a huge resistance to inter-disciplinary research. This view crystallized for me last fall, when I attended a two-day workshop on “the Future of Software Engineering”, but came away very disappointed (my blog post from the workshop captured this very well).
I should be clear, I don’t mean to write off the entire community – there’s some excellent people in the ICSE community, doing fascinating research – many of them I regard as good friends. But the conference itself seems ever less relevant. The keynote talks always suck. And the technical program tends to be dominated by a large number of dull papers: incremental results on unimaginative research problems.
Perhaps this is a result of the way conference publication works. Thomas Anderson sets out a fascinating analysis of why this might be so for computer systems conferences, in his 2009 paper “Conference Reviewing Considered Harmful“. Basically, the accept/reject process for conferences that use a peer-review system creates a perverse incentive to researchers to write papers that are just good enough to get accepted, but no better. His analysis is consistent with my own observations – people talk about “the least publishable unit” of research. The net result is a conference full of rather dull papers, where nobody takes risks on more exciting research topics.
There’s an interesting contrast with the geosciences community here, where papers are published in journals rather than conferences. For example, at the AGU and EGU conferences, you just submit an abstract, and various track chairs decide whether to let you present it as a talk in their track, or whether it should appear as a poster. Researchers are only allowed to submit one abstract as first author, which means the conference is really a forum for each researcher to present her best work over the past year, with no strong relationship to the peer-reviewed publication process. This makes for big conferences, and very variable quality presentations. Attendees have to do a little more work in advance to figure out which talks might be worth attending. But the perverse incentive identified by Anderson is missing all together – each presenter is incentivized to present her best work, no matter what stage the research is at.
Which brings me back to ICSE. Next year’s conference chairs have chosen the slogan “Sustainable Software for a Sustainable World” for the conference. An excellent rallying call, but I sincerely hope they can do more with this than most conferences do – such conference slogans are usually irrelevant to the actual conference program, which is invariably business as usual. Of course, the term sustainability has been wildly overused recently, to the point that its in danger of becoming meaningless. So, how could ICSE make it something more than a meaningless slogan?
First, one has to acknowledge that an understanding of sustainability requires some systems thinking, and the ability to analyze multiple interacting systems. The classic definition, due to the Bruntland Commission, is that it refers to humanity’s ability to meet its needs, without compromising the needs of future generations. As Garvey points out, this is entirely inadequate, as it’s impossible to figure out how to balance our resource needs with those of an unknown number of potential future earthlings. A better approach is to break the concept down into sustainability in different, overlapping systems. Sverdrup and Svensson do this by breaking it down to three inter-related concepts: natural sustainability, social sustainability, and economic sustainability. Furthermore, they are hierarchically related: sustainability of social and economic activity are constrained by physical limits such as thermodynamics and mass conservation (e.g. forget a sustained economy if we screw the planet’s climate), and economic sustainability is constrained by social limits such as a functioning civil society.
How does this apply to ICSE? Well, I would suggest applying the sustainability concept to a number of different systems:
- sustainability of the ICSE community itself, which would include nurturing new researchers, and fixing the problems of perverse incentives in the paper review processes. But this only makes sense within:
- sustainability of scientific research as a knowledge discovery process, which would include analysis of the kinds of research questions a research community ought to tackle, and how should it engage with society. Here, I think ICSE has some serious re-assessment to do, especially with respect to it’s tendency to reject inter-disciplinary work.
- sustainability of software systems that support human activity, which would suggest a switch in attention by the ICSE community away from the technical processes of de novo software creation, and towards questions of how software systems actually make life better for people, and how software systems and human activity systems co-evolve. An estimate I heard at the CHASE workshop is that only 20% of ICSE papers make any attempt to address human aspects.
- sustainability of software development as an economic activity, which suggests a critical look at how existing software corporations work currently, but perhaps more importantly, exploration of new economic models (e.g. open source; end-user programming; software startups; mashups, etc)
- the role of software in social sustainability, by which I mean a closer look at how software systems help (or hinder) the creation of communities, social norms, social equity and democratic processes.
- the role of software in natural sustainability, by which I mean green IT topics such as energy-aware computing, as well as the broader role of software in understanding and tackling climate change.
A normal ICSE would barely touch on any of these topics. But I think next year’s chairs could create some interesting incentives to ensure the conference theme becomes more than just a slogan. At the session on SE for the planet that we held at ICSE 2009, someone suggested that in light of the fact that climate change will make everything else unsustainable, ICSE should insist that all submitted papers to future conferences demonstrate some relevance to tackling climate change (which is brilliant, but so radical that we have to shift the Overton window first). A similar suggestion at the one of the CHASE meetings was that all ICSE papers must demonstrate relevance to human & social aspects, or else prove that their research problem can be tackled without this. For ICSE 2012, perhaps this should be changed to simply reject all papers that don’t contribute somehow to creating a more sustainable world.
I think such changes might help to kick ICSE into some semblance of relevancy, but I don’t kid myself that they are likely. How about as a start, a set of incentives that reward papers that address sustainability in one of more of the senses above? Restrict paper awards to such papers, or create a new award structure for this purpose. Give such papers prominence in the program, and relegate other papers to the dead times like right after lunch, or late in the evening. Or something.
But a good start would be to abolish the paper submission process all together, to decouple the conference from the process of publishing peer-reviewed papers. That’s probably the biggest single contribution to making the conference more sustainable, and more relevant to society.
Well said Steve – I completely agree. I gave up on ICSE a while ago after the Edinburgh conference where I think there was 1 paper which I found interesting. I sometimes think I should give it another try but your post has convinced me that it isn’t worth the effort.
Basically, ICSE (and CHI and the rest of the high profile conferences) is a means by which people increase their probability of getting tenure or doing well in the research assessment processes. Very few of the papers have got anything to do with the kind of software engineering that my industrial friends are involved with. The academic research community simply does not realise how far behind the state of the practice it really is.
The best decision that ICSE’s steering committee could make to show that their interest in sustainability is more than a slogan is to decree that sustainability will be the conference’s theme from now into the foreseeable future. How else can it ensure that a researcher’s work in sustainability will be sustainable, and not just the flavour of the year?
When the zombie apocalypse comes, I intend to barricade my door with ICSE proceedings, and use TOSEM to light fires. Until then, I don’t find them to be of much use.
By the way, the ICSE could really push for sustainability by letting participants watch and also give talks online. For sustainability, we need to virtualize travel, moving kilobytes instead of kilograms as much as possible. Software engineers can do their part by developing software that helps, and demonstrating it at their conferences.
For more try:
• Remote conferencing, Azimuth Wiki.
If anyone has any other good ideas, please add them to this wiki page!