Nature – January 21, 2010 - An opinion article by Professor Willy Aspinall of Bristol University in the British science magazine Nature focuses on the need to use better techniques to harvest expert advice on risk and uncertainty—particularly in the study of climate change.
When politicians and other decisionmakers seek advice from experts, they hope to receive something unambiguous that they can act on. But as risk consultant Aspinall explains, in complex situations—assessing the risk of volcanic eruptions and earthquakes for instance—attempts to force a consensus can be counterproductive. It is better to quantify the uncertainty that exists and build it into the decisionmaking process. The way to do this, says Aspinall, is through “expert elicitation,” specifically by using a method developed by RFF Senior Fellow Roger Cooke, which weighs the opinion of each expert based on his or her knowledge and ability to judge relevant uncertainties. Some have called for the United Nations Intergovernmental Panel on Climate Change (IPCC) to use such methods.
Aspinall cites the use of the Cooke method during a potential volcanic eruption in Montserrat, West Indies, in the mid-1990s. Authorities on the island sought advice on whether and when to issue warnings, impose travel restrictions, or conduct evacuations. Using expert elicitation, a group of volcanologists provided useful guidance, including the percentage chance of a violent explosion, within two hours.
“When scientists disagree, any attempt to impose agreement will ‘promote confusion between consensus and certainty’,” Aspinall writes, quoting Cooke. “The goal should be to quantify uncertainty, not to remove it from the decision process. Of the many ways of gathering advice from experts, the Cooke method is, in my view, the most effective when data are sparse, unreliable, or unobtainable.”
Aspinall also notes that expert elicitation has been recommended as a tool for the IPCC “to ensure that areas of uncertainty poorly captured by models are better represented, such as whether the Greenland ice cap might melt.”
Excerpts from the Article
|
“Advice during an emergency is usually the responsibility of a chief scientist, with all the stresses that involves—including the pressure to be extremely cautious. There is a better way: pooling the opinions of a group of specialists. There are several methods of such expert elicitation, each with flaws. The traditional committee still rules in many areas—a slow, deliberative process that gathers a wide range of opinions. This has parallels with the scientific process itself. But committees traditionally give all experts equal weight (one person, one vote). This assumes that experts are equally informed, equally proficient and free of bias. These assumptions are generally not justified.”
“The Cooke approach is starting to take root in similar problem areas, including climate change impacts on fisheries, invasive-species threats and medical risks. In Canada, policy-makers considering the risks of variant Creutzfeldt–Jakob Disease infection through contaminated blood supplies are utilizing scientific advice from a Cooke elicitation. Participants found the elicitation “extremely useful,” and suggested that it would be helpful in assessing other public-health issues, including pandemic flu and vaccination risks.”
“Uncertainty in science is unavoidable and throws up many challenges. On one side, you may have scientists reluctant to offer their opinions on important societal topics. On the other, you may have decision-makers who ignore uncertainty because they fear undermining public confidence or opening regulations to legal challenges. The Cooke approach can bridge these two positions. It is most useful when there is no other sensible way to make risk-based decisions—apart from resorting to the precautionary principle, or, even less helpfully, evading an answer by agreeing to disagree.”
#####
