“New study finds cycling has $1.4 billion economic impact on Iowa each year.”
This was the headline for an article published in the Des Moines Register in January of this year. It was covering a study we did on spending by cyclists in Iowa.
“Study: Adult-Use Cannabis Legalization in Ohio Would Generate $260M in ‘Net Benefits for Society’”
This was the headline for an article in cannabis industry news website Ganjapreneur in 2023 on a study we did on a ballot initiative to legalize recreational marijuana in Ohio.
While these headlines seem deceptively similar, we used very different methodologies to come to these two numbers. Ultimately, these two statistics are referring to two different things entirely.
The former article was an approach called “economic impact analysis.” Economic impact analysis works by finding a base spending figure then estimating how that spending ripples throughout the economy. We do this using a concept called “economic multipliers.”
While it sounds technical, the intuition behind multipliers is simple. When you spend money at a bicycle store for example, the business spends that money on more bicycles and wages for its employees. Employees then spend that money on rent, groceries, transportation to work, and the bicycle wholesaler spends that money on buying bicycles from manufacturers and paying its employees. Then the bicycle manufacturer spends that money on employees and parts, and so on.
Researchers at the Bureau of Economic Analysis have analyzed spending patterns across the country and have used that data to estimate how much a dollar spent in one industry spurs new spending in other industries. We can use this to estimate how much spending in one industry ends up impacting other industries, how it grows wages, and how many jobs are supported by an industry.
The latter study on cannabis legalization works differently. For this study, we utilized a framework called “cost-benefit analysis.” Cost-benefit analysis concern more spending patterns, instead looking at the economy in a broad sense by including nonmarket impacts. Because analysts conducting cost-benefit analysis monetize outcomes that are not traded on markets, they do not have precise data to work with that they do when conducting an economic impact analysis. Often, cost-benefit analysis involves using studies on the impacts of policies in certain places and making adjustments based on local conditions to estimate where the impacts of a policy could be elsewhere.
Early childhood education is a great example of this. Most of the work done by Nobel Laureate James Heckman and Leading Economic Development Economist Timothy Bartik relies on studies such as the Perry Preschool Project and the Abecedarian Project. These experiments were randomized controlled trials conducted in the 1960s and 1970s with follow-up surveys collected in the decades that followed. These studies allowed researchers to estimate the long-term impacts of programs by comparing a treatment group to a control group that did not participate in the early childhood education program. We assume that the impacts of these programs in the 1960s and 1970s through today will be similar to the impacts of programs implemented today 50 and 60 years into the future.
This isn’t to say that economic impact analysis does not come with its own assumptions. Multipliers an analyst uses today are based on economic conditions a few years old. Upheavals like the COVID-19 pandemic, Brexit, or new tariffs across the economy can reshape the economic landscape in a short amount of time, making data collected just a few years ago less informative to analysis done today. But they still provide a starting place for analysts and decision makers.
One thing economic impact analysis has come under fire for over the past decade is its use in promoting subsidies for stadiums.
Often sports teams justify public subsidies with economic impact analysis studies showing hundreds of millions of dollars of economic impact associated with the investments. These are often the result of restaurants and bars developing around a sports stadium after construction, generating new economic activity where there was no activity before.
The reason economists are skeptical of these studies is the lack of accounting for opportunity costs. Economic impact analysis, by its design, only tells an analyst what the benefits of economic activity or a project are. They do not address costs.
The main costs economists worry about around stadiums is the entertainment that they displace. Since people are spending money at restaurants and bars in close proximity to a stadium, they are not spending their money elsewhere in the city. Since entertainment budgets are not very flexible (people do not suddenly stop spending money on rent, groceries, and gas to go to a Browns game because of a new stadium), new developments usually means displacement of economic activity elsewhere.
This does not address other costs, such as the tax drag created by financing the subsidies and potential social costs such as violence associated with bars in proximity to the stadium.
This is where cost-benefit analysis can be a more useful tool for analyzing the impact of a project. A well-designed cost-benefit analysis will give a policymaker an understanding of the major economic impacts of a policy–positive and negative.
This does not mean a cost-benefit analysis is superior to an economic impact analysis. They answer different questions. An economic impact analysis answers the question of “how big this project or sector is in the grand scheme of the economy.” A cost-benefit analysis answers “what the net impact adoption of a policy or program will have on the economy.”
Economic impact analysis can confuse the public if not explained clearly. Even with all the work we have done to explain the limitations of economic impact analysis and what exactly it tells us, reporters are human. They still make simple mistakes like comparing spending on a program to its economic impact and trying to make back-of-the envelope return on investment calculations. What you need to keep in mind is this: economic-impact analysis helps you understand how big an impact a sector or policy is, not whether its benefits outweigh its costs.