- Paper on SSRN
- LPP Working Paper N° 2-2022. Second prize winner of LPP's Cost-Benefit Analysis Writing Competition (URL)
Abstract
Cost-benefit analysis embodies techniques for the analysis of possible harmful outcomes when the probability of those outcomes can be quantified with reasonable confidence. But when those probabilities cannot be quantified (“deep uncertainty”), the analytic path is more difficult. The problem is especially acute when potentially catastrophic outcomes are involved, because ignoring or marginalizing them could seriously skewing the analysis. Yet the likelihood of catastrophe is often difficult or impossible to quantify because such events may be unprecedented (runaway AI or tipping points for climate change) or extremely rare (global pandemics caused by novel viruses in the modern world). OMB’s current guidance to agencies on unquantifiable risks is now almost twenty years old and in serious need of updating. It correctly points to scenario analysis as an important tool but it fails to give guidance on the development of scenarios. It then calls for a qualitative analysis of the implications of the scenarios, but fails to alert agencies to the potential for using more rigorous analytic techniques.
Decision science does not yet provide consensus solutions to the analysis of uncertain catastrophic outcomes. But it has advanced beyond the vague guidance provided by OMB since 2003, which may not have been state-of-the-art even then. This paper surveys these developments and explains how they might best be incorporated into agency practice. Those developments include a deeper understanding of potential options and issues in constructing scenarios. They also include analytic techniques for dealing with unquantifiable risks that can supplement or replace the qualitative analysis currently suggested by OMB guidance. To provide a standard framework for discussion of uncertainty in regulatory impact analyses, the paper also proposes the use of a structure first developed for environmental impact statements as a way of framing the agency’s discussion of crucial uncertain outcomes.
