In my work doing evidence-based policy analysis and extolling the virtues of cost-benefit analysis, I’ve had people ask me a question from time to time that seems quite esoteric and a little silly, but that any true cost-benefit analyst should care about: what is the return on investment for cost-benefit analysis? In other words, has there ever been a cost-benefit analysis done on cost-benefit analysis?
For the life of me, I have been unable to find a cost-benefit analysis done on cost-benefit studies. Cost-benefit minded policymakers, though, should be interested in finding an estimate of what the value of a cost-benefit analysis is. So let’s go through the exercise of what a cost-benefit analysis of cost-benefit analysis would look like.
First, let’s estimate the cost of a cost-benefit analysis. I am going to err on the more expensive side in order to more conservatively estimate the value of a cost-benefit analysis. According to Glassdoor.com, the average policy analyst in the United States earns a salary of about $68,000. Using a Boston Business Journal article from 2005 and adjusting to 2019 dollars, a high-end estimate of $27,000 for benefits and employment taxes (40% of salary) and an additional $2,300 for office space and equipment brings the total annual cost of a policy analyst to about $97,000. Let’s assume on the conservative side that the policy analyst will complete a cost-benefit analysis every quarter. This means that a cost-benefit analysis will cost, on average, about $24,000.
Now let’s estimate the worst-case scenario. This is a scenario where a policymaker commissions a cost-benefit study and decides to make the same decision she made before the study. Assuming there is no value to the knowledge that the policymaker is right, those $24,000 would go to “waste.” There would be $24,000 in accounting costs and zero dollars in benefits. From an economic perspective, we can use a high estimate for the marginal excess tax burden, again erring on the conservative side. Say the study was financed from income tax dollars and those tax dollars had an especially high drag on economic productivity, say reducing economic activity among those taxed by 43%. This means that the total social cost of a cost-benefit analysis would be about $10,000.
Let’s look at the flip side: an extremely successful cost-benefit analysis. This would mean that a cost-benefit analysis uncovered an especially inefficient program and identified an especially efficient alternative, which the policymaker then reallocated dollars towards. The most extreme example I could construct is pulled from programs listed in the Washington Institute for Public Policy (WSIPP)’s Benefit Cost Database. Let’s say there was a small “Fast Track prevention program” with 16 children participating. This program would have net costs of about $1.5 million to run according to WSIPP analysis (16 children x $91,000 net costs per participant). Say a policy analyst identified the loss and recommended the dollars be shifted to deploy an additional police officer using hot spot strategies ($430,000 in net benefits according to WSIPP) and this recommendation was adopted by the policymaker. This would result in a net benefit change in the policy of a total of $1.9 million.
This would mean the $10,000 economic cost of the study would be totally swamped out by the benefits of this program. As a matter of fact, with this ROI, the next 180 cost-benefit analyses could be “failures,” not showing new results or not leading to policy changes for other reasons, and this study alone would still lead to net benefits for the analysis program as a whole.
Let’s take a more moderate example. Let’s say a department of Juvenile Justice was reallocating funds towards higher-yield programs. What if a cost-benefit analysis found that an adolescent diversion system with services ($2,900 in net benefits per participant according to WSIPP) was having no better impacts than one without services ($9,500 in net benefits per participant according to WSIPP). This means a total of $6,700 of net benefits would be realized for every juvenile moved from the program with services to the program without, which means that if at least two juveniles were moved, the $10,000 cost-benefit analysis would have net benefits.
It’s worth noting here that the marginal cost of the implementation of cost-benefit insights is extremely low. While moving one participant from a services program to a standard program after a cost-benefit analysis would have net costs of $3,800, moving two participants would have net benefits of $2,900, moving three participants would have net benefits of $9,600, and so on.
With these exercises in mind, we can draw four rules for policymakers determining whether cost-benefit analysis is worthwhile in a given situation.
Use cost-benefit analysis when policymakers are open to changing their minds. If policymakers have a policy plan in place already and are unwilling to adjust it, a cost-benefit analysis is a waste of money and will not lead to policy change or net benefits.
Use cost-benefit analysis when current program effectiveness is suspect or alternatives may yield better returns. Opportunities to substitute low-benefit or negative-benefit programs for high-benefit programs can quickly justify the cost of a study.
Conduct cost-benefit analysis on large programs. Even the smallest changes identified by a cost-benefit analysis of a large program will justify the cost of the study. While small programs can justify the cost of a study with large enough changes, if total net program costs or benefits are expected to be only a few thousand dollars or less, cost-benefit analysis may not be a wise use of resources.
Conduct a variety of cost-benefit analyses to “diversify your analysis portfolio.” As the Fast Track example above shows, a killer cost-benefit analysis can justify scores of duds. If a policymaker is open to implementing more efficient policy, she should commission a number of analyses of specific, pertinent policy problems and empower staff to act on recommendations that come from these analyses.
Cost-benefit analysis isn’t going to pay off every single time it is carried out. But more cost-benefit analysis applied to a variety of large, previously unstudied programs will lead to better use of limited public resources with the right leaders at the helm. And this is why I’m spending my time on this sort of work, because when cost-benefit analysis is done right, it has the potential to be the most effective public policy intervention of them all.
Rob Moore is the Principal for Scioto Analysis.