Examining the effectiveness of risk elicitations: comparing a deliberative risk ranking to a nationally representative survey on homeland security risk

Comparing homeland security risks is a challenging example of comparative risk assessment. One methodology designed for comparing diverse risks of this sort is the Deliberative Method of Ranking Risks. Previous studies have evaluated the utility of the method in absolute terms, examining informed rankings at various stages of the process; this paper represents the first known approach to compare the method relative to another approach. As the Deliberative Method for Ranking Risks is designed to engage deliberative System 2 thought, we compared the rankings from the method to those from a nationally representative survey (RAND's American Life Panel) that engages experiential System 1 thought. We find evidence that the Deliberative Method for Ranking Risks works as intended, developing more informed rankings with less evidence of bias. The Deliberative Method for Ranking Risks can be a useful improvement for ad hoc comparisons of risk in the homeland security domain.