The economics of plowing streets

As a Minnesotan, I love the winter. To enjoy living in a place that can be this inhospitable, you have to find ways to enjoy being outside when it's cold. For me, that usually means being on the ice, whether it's skating or playing broomball. 

One thing I don’t like about Minnesota is how much snow impacts my experience on the roads. 

The city of Saint Paul has notoriously bad plowing, particularly on smaller side streets in residential neighborhoods. Growing up here, I can’t remember a winter where the street outside my parents’ home was ever visible in the middle of winter. From the time the first major snowstorm hit until the snowmelt during the spring, that street had an impenetrable layer of snow. Saint Paul residents get especially worked up about this issue because our neighbors in Minneapolis seem to have figured out how to effectively plow their streets.

As we begin another winter of poorly plowed roads, the question came across my mind: how much is it worth to have well plowed roads? 

I understand that for the city of Saint Paul to improve their road plowing system there would be costs. But do we know whether or not it would be worth it?

I did a little bit of cursory research, and I came across this report from 2014 about the lost economic activity that results from a snowstorm. In one of their key takeaways from their executive summary, the authors note “The economic impact of snow-related closures far exceeds the cost of timely snow removal. Although states and localities may be hesitant to expend significant upfront resources in the short-term, the long-term payoff more than justifies the expense.”

For Minnesota, the authors estimate that a major snowstorm would cost the state over $167 million (~ $232 adjusting for inflation) in lost economic activity. This report is looking at situations where conditions are so bad that stores aren’t open, which clearly have negative economic consequences. To Saint Paul’s credit, when major snow storms happen they do a good job of getting major roads open quickly. Businesses rarely have to stay closed as a result.

Unfortunately, this report doesn’t really cover the costs associated with bad plowing of smaller streets. How much economic activity is lost because people don’t want to leave their houses when it’s annoying to go a few blocks on poorly plowed roads to get to the main streets? How much longer are people’s commutes every day? 

Let’s do some back of the envelope math:

Say a person commutes every weekday. There are 13 weeks between the start of December this year and the last week of February which adds up to 65 commutes. If the poor road conditions add five minutes each direction, that adds up to over 10 additional hours of commuting over the main winter months. Maybe it’s unrealistic to think that better plowing could save that much time per commuter, but it still adds up quickly. 

I would love to see a more detailed analysis of how city plowing policies impact the economy. Maybe it would be possible using something like the American Time Use Survey if there was some sort of natural experiment where a city changed their plowing methods. Plowing is an important public service, and it would be good to know exactly how much value it provides.

Is Ohio still affordable?

According to politicians across the spectrum, there is currently an affordability crisis in the United States. Consumer prices across most categories of goods and services continue to rise month to month, though they have slowed since the height of inflation in 2022. The cost of housing is one of the biggest factors driving increases in cost of living across the country, with a median of 31% of household income for renters being spent on rent and a median of 21% of household income for homeowners being spent on mortgages in 2024. 

As housing prices continue to rise in 2025, more people are falling behind on paying their utility bills. Between 2024 to 2025, annual past due balances to utility companies increased by 9.7%, while monthly energy bills increased by 12%. If renters and homeowners hope to follow the age-old guideline of spending no more than 30% of income on housing–a threshold the median renter already exceeds–income must follow suit and rise alongside the cost of living.

In Ohio, the cost of living continues to rise as well, though there may be some upside compared to the nation as a whole. To afford a modest two-bedroom apartment in 2025, an Ohio resident must make at least $22.51 an hour working full-time, an 8% increase from last year. Simultaneously, the average Ohio renter earns just $18.62 an hour, and homelessness in Ohio has risen by 3% since 2024 to 11,759 people.

In Ohio, the cost of living continues to rise as well, though there may be some upside compared to the nation as a whole. To afford a modest two-bedroom apartment in 2025, an Ohio resident must make at least $22.51 an hour working full-time, an 8% increase from last year. Simultaneously, the average Ohio renter earns just $18.62 an hour, and homelessness in Ohio has risen by 3% since 2024 to 11,759 people.

If home prices are so much lower in Ohio than most of the United States, how do actual monthly housing costs compare? We can use data from IPUMS, a database that provides American Community Survey microdata, to answer this question. In 2023, the median gross rent including utilities across the United States was about $1,448 per month. How do monthly housing costs in Ohio compare to the nation as a whole?

Compared to the national average, monthly housing costs across eight of the largest counties in Ohio are all lower than the national average. My economic intuition would be that Ohio’s relatively low cost of housing compared to the national average, especially in major cities such as Cleveland (Cuyahoga County), Cincinnati (Hamilton County), or Columbus (Franklin County), would incentivize workers and families nationwide to move into Ohio. On housing alone, the median renter could save $2,400 to $4,800 per year, all while still living near a major city. 

However, data from the Census Bureau tells us the opposite story: between 2020 to 2024, Ohio experienced a negative net migration rate, ranking 39th out of net domestic migration rates across all 50 states. If states like Ohio are so much cheaper in a time of an affordability crisis, why aren’t people moving to them?

One possible explanation could be that lower housing costs are a facade for higher commuter costs. In 2024, a typical household in Ohio spent about 27% of their yearly income on transportation costs, mostly from car ownership, which is 5 percentage points higher than the national average and 12 percentage points higher than the recommended threshold of affordability. In Ohio, the median household income in 2024 was $72,212. Let’s say an average household moves from an average city in the United States to a city with lower housing costs in Ohio. They may see up to $4,800 dollars back in their pockets from the reduced cost of housing, but they’ll also be spending around $3,600 more per year on transportation costs.

There are a range of other price differences that impact affordability across the United States. For example, some people live in food deserts–areas where access to grocery stores and nutritious food may be limited. In these cases, consumers may have to choose between paying low prices for food or getting their proper nutrition. Another example is childcare affordability: some states, like Ohio, have far fewer subsidies for public childcare, making the cost of childcare a larger burden on household budgets. Or, some states and municipalities could have lower or higher income tax, sales tax, or property tax. In many cases, affordability differences like these muddy the waters of where actually is affordable in the United States– and which cities are the most affordable for certain demographics.

The regional price parity index from the Bureau of Economic Analysis provides reliable estimates of the differences in cost-of-living across different states. States like Arkansas, Mississippi, and South Dakota are among the most affordable, while states like California, New Jersey, and Hawaii are among the most expensive. 

Regardless of how cost-of-living differences look statewide, the nuances of how an individual family spends their money are an important factor in how affordable or costly living in a particular city is. And we haven’t even touched on the opposite side of the equation here: income. No matter how affordable a state is, employment opportunities with high incomes will drive migration trends as much or even more than affordability. That is why Arkansas, Mississippi, and Ohio have not become magnets for migration: despite affordability advantages for these states over coastal states, employment is still king when it comes to the economics of migration.

What should Ohio’s poverty line be?

Over the weekend, an article by a financial advisor named Michael Green made headlines in a number of prominent media sources.

In this article, Green argues that the federal poverty line for a family four shouldn’t be $32,000 as the U.S. Department of Health and Human Services designates it, but should instead be $140,000.

Green comes to this conclusion by collecting average costs for a range of goods and adding them up.

This is similar to the approach taken by the Massachusetts Institute of Technology Amy Glasmeier in her Living Wage Calculator.

Her number for Ohio comes out to $38.47 per hour, which equals about $80,000. So considerably more than the federal poverty line, but considerably less than what Green comes up with.

There are reasons to be skeptical about this approach to measuring poverty.

A central question here as a former philosophy undergraduate and a current poverty analyst is “what is poverty?”

Whether we’re thinking about poverty as a measure of absolute deprivation connected to some sort of sense of biological needs or if we’re seeing it as a measure of relative deprivation connected to social needs, using average expenditure on a wide range of goods as the sole determinant of the threshold is a problematic way to define poverty.

The most widely-accepted measure of poverty by poverty researchers is the Supplemental Poverty Measure.

This is a poverty measure that uses spending as a baseline for its measurement, but instead of using average spending as the baseline, it measures how many households are below a certain percentage of average spending.

Its threshold for a family of four in 2024 was about $39,000. So still considerably higher than the $32,000 of the official poverty threshold, but far below the MIT and Green calculations.

In the Organisation for Economic Co-operation and Development, an international organization for development of economic policy, poverty is defined at 50% of median household income.

This is a handy measure when measuring across countries due to simplicity, and also captures the essence of practical poverty policy across countries: how many people fall far from the norm when it comes to household resources.

Using this measurement, a poverty threshold for Ohio would be about $36,000, half of the median household income at $72,000.

How do we make sense of these different measures? Is the poverty threshold $32,000, $39,000, $80,000, or $140,000?

As a policy analyst, my answer to this question is this: which of these actually fit with our understanding of how poverty works in the United States?

If you surveyed people at $75,000, would they, on average, consider themselves to be living in poverty?

If not, then an $80,000 poverty threshold is defining that family into a financial position they don’t consider themselves to be in. Moreover, $140,000 stretches the definition of poverty to absurdity.

Another test: what is the point of the poverty measure?

According to the Census Bureau, there are 1.2 million Ohio residents living under the official poverty threshold and 1.1 million Ohio residents under the supplemental poverty threshold.

If the goal of poverty policy is to help the people who need it most, shouldn’t this population of one million plus warrant more attention than the over 9 million who make under $140,000?

Poverty measure is an inexact science.

But widening the scope of poverty to include four out of five U.S. families is a great way to miss out on the plight of those struggling the most in the state.

Measurements like relative poverty measures or the Supplemental Poverty Measure give policymakers a much clearer picture of poverty than summing average costs of goods across society.

New Scioto Analysis Report Finds Raising Minimum Wage to $15 Improves Health Outcome in Oklahoma

A new report conducted by Scioto Analysis and released by This Land Research and Communications collaborative reveals that raising Oklahoma’s minimum wage to $15 per hour would lead to improved health outcomes across the Sooner state by raising household incomes, reducing financial stress, supporting healthier behaviors, and increasing access to care. 

Highlights from the report show the benefits of raising the minimum wage to $15 an hour include:

• Preventing approximately 400 deaths annually, including 240 infant deaths, 26 cardiovascular deaths, and 19 suicides.

• Improvements of self-reported health status for over 31,000 Oklahoma residents.

• Reducing unnecessary emergency room visits by more than 6,000 annually, saving approximately $5 million for Oklahoma taxpayers every year.

• Expanding access to healthcare, especially in low-income and rural communities.

“Oklahoma consistently ranks near the bottom in health outcomes, including life expectancy and chronic disease,” Rob Moore, principal analyst at Scioto Analysis, said. “Our policy analysis shows that raising wages could be an effective public health tool—one that saves lives, reduces taxpayer dollars spent on healthcare like unnecessary emergency room visits, and narrows persistent health disparities.”

The report also highlights how a higher minimum wage could reduce financial stress, support healthier behaviors, and allow more Oklahomans to access routine and preventive care, especially in areas where high out-of-pocket costs and healthcare access currently limit treatment options.

“While a minimum wage policy alone cannot solve all health challenges facing Oklahoma families, our analysis suggests that improving wages for working Oklahomans can directly improve health, reduce taxpayer dollars spent on healthcare costs, and generate significant social and economic benefits.” Moore concluded.

The full report, Minimum Wages and Health Outcomes in Oklahoma, is available here.

Scioto Analysis releases 2026 State Handbook of Cost-Benefit Analysis

This morning, Scioto Analysis released the 2026 edition of the State Handbook of Cost-Benefit Analysis. The State Handbook of Cost-Benefit Analysis is a handbook for state analysts interested in adding cost-benefit analysis to their analytical toolkit.

Cost-benefit analysis is a tool to evaluate the social costs and benefits of public policy through an economic lens. The 2026 edition provides analysts with the history and theory behind cost-benefit analysis and it provides comprehensive guidance and examples for conducting cost-benefit analysis. Designed for administrators, policymakers, and engaged citizens, the Handbook walks through how to establish a baseline, construct alternatives, establish a standing, complete quantification and monetization, categorize costs and benefits, discount impacts appropriately, conduct sensitivity analysis, and communicate findings. 

The 2026 edition focuses on the analytical process of conducting cost-benefit analysis and expands upon quantification, monetization, and communicating results. Cost-benefit analysis is a useful tool, even for administrators, policymakers, or citizens with limited analytical resources. 

Cost-benefit analysis is used commonly at the federal level but is still underutilized at the state level. Analysts who are interested in promoting evidence-based decision making among policymakers and administrators should consider using the State Handbook of Cost-Benefit Analysis to adopt cost-benefit analysis in their analysis.

Economic development and the “winner’s curse”

I feel like I’m a broken record sometimes talking about how much Planet Money sparks ideas for me to write about public policy. But, they’ve done it again. And this time it is around a concept called the “Winner’s Curse.”

Planet Money’s recent story was focused on the re-release of Richard Thaler’s The Winner’s Curse, a 1991 book on a concept he had pieced together in his work pioneering the field of behavioral economics. Since this book was published, Thaler has won the Nobel Prize in Economic Sciences and the field of behavioral economics has gone from fringe to mainstream.

Thaler’s central concept in “The Winner’s Curse” is that there are certain times that bidding wars can push people into irrational behavior. Theoretically, each person should have a specific willingness to pay for a good in a bidding war. Theoretically, a bidding system creates an efficient system of allocation where people increase their bids until their willingness to pay, then do not bid anymore. This means that the person with the highest willingness to pay will then stop on their last bid, receiving the good at the lowest price possible according to the market.

So let’s say you have a “priceless” Dalí painting. Adam really wants it and is willing to pay $3.5 million. Beth wants to buy it as well and is willing to pay $2.9 million. Adam initially bids $2 million. Beth increases the bid to $2.5 million. Adam increases it to $3 million. Beth does not bid any longer, since the price has exceeded her willingness to pay. So Adam buys it at $3 million, receiving $0.5 million in consumer surplus since he would have paid up to $3.5 million.

A problem with this model is the following: why do sellers put their goods up to auction? Ideally, a seller wants to get the most producer surplus possible, which means pushing the buyer as close to her willingness to pay as possible. What benefits does an auction bestow the seller?

Well one problem is an information problem. You will notice that many auctions are for goods with indeterminate value. There is no market for rare Dalí paintings, so we can’t determine what the market value is. So having buyers go to auction over it helps solve that problem.

Another explanation, though, is that auctions may do a different service for the seller. It may induce the buyer to act irrationally.

What if Beth doesn’t stop at $3 million, but instead jumps to $3.5 million to top Adam? Then Adam jumps to $4 million to top her? Suddenly, each of the bidders are above their willingness to pay. Suddenly, the person who wins will end up with the curse of negative consumer surplus from something they will buy and the seller will run away with a bigger payment than they ever could have got without the auction.

This is the curse: how auctions can lead us to act irrationally by playing on our competitive spirits. The win is fleeting: ultimately, the prize is not worth what we pay for it.

What this reminded me of was economic development.

We’ve all heard about it: the exorbitant packages that are given away in tax incentives to lure businesses across borders. When Amazon announced its “HQ2” project, it became the ultimate prize for economic development professionals. Local governments across the country scrambled to offer whatever they could to get into the running, with the winners ultimately stuck with the tab.

The leading national researchers on economic development incentives have concluded that somewhere between 75% and 98% of projects would have been sited in the same place they ended up going to without economic development incentives. That means that in somewhere between three-quarters to nearly all projects, the market clearing price for incentives is zero. All that money that is being spent by most economic development projects is pure producer surplus for the people selling their new development.

Why do cities keep suffering the winner’s curse? A few reasons.

The game is set as a competitive game for economic development professionals. Economic development professionals don’t get rewarded for passing up projects that get too expensive, they only get rewarded when they land projects. This creates an incentive for them to overpay for projects and promise more in the form of deferred tax revenue than the community’s willingness to pay for the project.

A reason this materializes is due to lack of transparency in projects. Often, in the name of securing a “competitive edge,” economic developers who are in charge of creating incentive packages are shielded from the public. This allows them to privilege their own interests over the interests of the public. They will not ultimately pay for giving away too much in deferred taxes: the public will. They will pay, though, if they do not win the development project. This worsens the principal-agent problem between economic development professionals and the public and makes it more likely for developers to fall into the winner’s curse.

This insulation from the public also worsens another problem: a lack of clear, stated goals. If you go into an auction and do not know what your willingness to pay for that Dalí is, you become a sucker for the winner’s curse. You will make “winning” the goal instead of maximizing consumer surplus. Often, economic developers do not have a walk away price or a goal other than to win the economic development project. This predictably leads to them becoming victims to the winner’s curse.

Economic development is not about “winning,” but by pitting different sites within different communities against each other, corporations are able to utilize auction dynamics that ultimately get communities to jack up their offer prices to the point where they exceed the point of marginal social benefits. In the economic development world, the winner’s curse is real. Only by rooting economic development incentives in a framework of social value will economic developers be able to sidestep the winner’s curse and focus on projects that maximize social value. Until then, the economic development world will continue to bear the curse.

Does forecasting actually work?

Most of policy analysis and indeed most of the work we do at Scioto Analysis is forward-looking. We sometimes look back at policies that have been put into place to determine what effects they had (which is sometimes called “evaluation” and sometimes called “ex post analysis”), but in order to help policymakers make better decisions, we need to be able to give them information about what will happen if they choose to enact a given policy.

We call this “forecasting,” but if you want to be less technical, we’re trying to predict the future. We have tools that give us confidence that our predictions are better on average than just randomly guessing about the future, but at the end of the day we are making a claim about something that has not happened yet. 

One question you might have about forecasting is straightforward: does it work? If people are making so many predictions, why don’t we just do some evaluation and see if they are accurate or not?

A new working paper from the National Bureau of Economic Research tries to answer this question. 

In this study, economists looked at 100 social science projects which made tens of thousands of forecasts between 2020 and 2024. They wanted to find out whether or not these forecasts were successful at predicting what would happen in the future, and if they were able to identify any characteristics that made people better or worse at forecasting. 

The good news is that, in general, the forecasts they looked at had predictive value. There was a tendency for forecasts to overestimate the treatment effect, but in general, a forecast was able to provide meaningful insight about the future. 

This study also supported the “wisdom of the crowd” hypothesis. When the researchers found multiple forecasts about the same topic, then the average of those forecasts would be a better predictor than any one by itself. 

When looking at the quality of predictors, the researchers found that academics (professors and top-ranking PhD students) performed better than non-academics. Since results tend to be overestimated, this generally means that these academics are finding smaller effect sizes, which might explain why they have the perception of being cautious. In a similar vein, the researchers found out that people who were more confident in their estimates tended to perform worse than people who were less confident. 

Overall, I think this is a success story for forecasters. When I think about the information a policymaker tends to want, the first thing we need to do is make sure our effect direction is correct. If we say a policy is going to grow the economy, then it better not shrink it. 

Of course, we want to be as accurate as possible and any good policy analysis should be rigorous, but getting the direction correct already gives policymakers information to make their decisions with. 

At the end of the day, the goal of analysis is to inform better decision making. This paper suggests that in general, social scientists are doing a good job at getting the main idea right. We should probably be skeptical about the magnitude of some effects, but overall, research helps us understand what the future might hold.

What would a measles outbreak do to your community?

Last week, the Centers for Disease Control and Prevention released new information on its website reviving a widely-debunked claim that vaccines could be linked to autism. U.S. Department of Health and Human Services Secretary Robert Kennedy subsequently took responsibility for the website change change.

From a public health standpoint, the danger of misinformation around vaccination comes from the possibility that vaccination rates will fall. Vaccination is one of the best tools the public has to prevent the incidence of life-threatening diseases and reduced rates of vaccination can lead to diseases being reintroduced into local populations.

Despite the Centers for Disease Control and Prevention’s new website information, the site still hosts an useful tool for answering questions around vaccination and one major disease–measles. The Centers’ Measles Outbreak Simulator lets anyone simulate a measles outbreak in a community and what different vaccination rate and policy interventions would do to stymie the spread of the disease in the public.

Let’s look at a community of 100,000 people to see what the simulator tells us.

First, let’s look at the best-case scenario. Currently, West Virginia’s measles vaccination rate is over 98%, the highest in the United States. If a community of 100,000 with 98% vaccination rate has a measles case, the simulator expects only one measles case will occur and no one will be hospitalized. Widespread vaccination works. If we drop this number to a 95% vaccination rate, the goal rate for vaccination and the rate we see in Tennessee, we only get one additional case and a single hospitalization.

Most states are below this rate, though. The median U.S. state, South Carolina, is at 92% measles vaccination. The introduction of a single measles case in a 92% vaccinated community, though, would only lead to a total of five measles cases in our simulation community, and only a single hospitalization.

Things start to pick up once you drop below this median number, however. The 25th percentile of states (states like Arizona and Hawaii) are vaccinated at under 90%. A single measles case introduced in a community of 100,000 with 90% vaccination coverage would blossom into a total of 5,800 cases and about 860 hospitalizations. If the death rate mirrors 2025 U.S. death rates, an outbreak like that would lead to about 10 deaths.

Things get even more dangerous once you drop below 80%, which only the state of Idaho has done so far. A community of 100,000 with 80% vaccination rates would see over one in five people infected with measles, about 21,000 cases and 3,100 hospitalizations. At current U.S. measles death rates, this would also lead to about three dozen deaths.

Okay, so let’s say we get to this disaster scenario and a community at 80% vaccination rate has a measles case introduced. What can they do? Well, the simulator from the Centers for Disease Control and Prevention also lets us test different interventions.

One intervention is to catch up with vaccinations. If the community starts a vaccination campaign a few days after the first case is caught and gets 75% of unvaccinated people vaccinated over the course of a few weeks, total measles spread will drop to 16, a 99% reduction and three dozen lives saved. Even vaccinating half the unvaccinated population would lead to a 57% reduction in measles cases and save 21 lives. A more conservative campaign that only gets 20% of the unvaccinated population vaccinated would cut infection rates by 21% and save seven lives. 

Less aggressive vaccination campaigns that vaccinate people more slowly or begin later are also effective at reducing measles spread. Even a five-month vaccination campaign started at the beginning of infections and reaching a total of 20% unvaccinated population vaccination rate would have similar results to stymie the spread. Similarly, a vaccination campaign that begins a few months after the first measles case is similarly effective to one that starts a few days after the first measles case. This means that the public health response should not feel like they “missed the boat” if they did not respond right away: widespread vaccination can still make a difference as long as it is implemented within a few months of the first recorded cases.

What if a community isn’t willing to turn to widespread vaccination? Another option is isolation of measles cases. The problem with this intervention is how quickly measles spreads. If 50% of people who contract measles isolate for a whole year each, only 9% of measles cases would be prevented. These durations of isolation are likely unachievable, making this an unviable strategy for making strong gains against measles.

Another option is quarantine. A communitywide quarantine would have to last more than a month and achieve 50% adherence to significantly reduce measles rates. A community of 100,000 would have to work very hard to achieve these adherence rates for this duration to stop the spread of measles in their community.

What I learned from this simulator is this: vaccination is still king when it comes to measles. As much as a barrier vaccine hesitancy is, isolating people who contract the disease from the public for a year or shutting down a community for over a month are each less practical and more disruptive than community-wide vaccination campaigns.

Which brings us back to the public policy issue at hand. The Centers for Disease Control shares in their own simulations that vaccination is the best way to prevent spread of dangerous communicable diseases. Scaled up to a state like Ohio, vaccination rates dropping from its current 89% to 80% could mean thousands of measles deaths per year. State health departments have a strong interest in combatting sources of misinformation that lead to vaccine hesitancy among the general public. If people skip out on vaccination, it puts them in harm’s way, and it also endangers their neighbors. Preventing states from backsliding on vaccinations means saving lives, preserving individual choice, and keeping the economy on its rails at the same time.

Survey: Majority of Ohio economists agree new Ohio affordable childcare program will improve educational outcomes for children.

In a survey released this morning by Scioto Analysis, 13 of 19 economists agreed that the new Child Care Cred affordability program announced by Ohio Governor DeWine in September 2025 will improve educational outcomes for children in Ohio. The Child Care Cred affordability program will allow families whose incomes are between 200% to 400% of the federal poverty line to share childcare costs with their employers and the state. Through the program, employees will cover 40%, employers will cover 40%, and the state will cover the remaining 20% of childcare costs.

Respondents voiced opinions that subsidized childcare can improve educational performance later in life, free up money for parents to spend on other resources for their children, and help improve social mobility among low-income children and families. As Jonathan Andreas of Bluffton University commented, “Most children in America have gotten an educational boost by going to professional childcare centers because the average American kid grows up [in a] low-income, low-education household where they get less mental stimulation than they would get in a professional childcare center”. 

Michael Jones of the University of Cincinnati disagreed with the consensus, suggesting that if the Child Care Cred affordability program pushes mothers back to work more quickly, child educational outcomes may suffer. He notes, “A growing body of research shows that mothers who spend more time at home with their young children, rather than rushing back to work, see better outcomes for both themselves and their children. [...] In addition, children who spend more time with a parent in their early years (rather than in institutional care) can realize increased cognitive skills and stronger emotional development”. 

While most economists agreed that the Child Care Cred affordability program would improve educational outcomes for children, they had mixed opinions on how work requirements for public childcare programs would impact economic outcomes like unemployment rates. In Ohio, parents must now work at least 33 hours per week to qualify for full-time publicly funded childcare benefits. Three economists agreed that work requirements for publicly funded childcare will reduce unemployment rates, nine economists disagreed, and nine economists were uncertain. David Brasington of the University of Cincinnati disagreed that work requirements for publicly funded childcare will reduce unemployment rates, explaining that, “Almost all daycare subsidy recipients use the daycare money to free up time to engage in market work already”.

The Ohio Economic Experts Panel is a panel of over 30 Ohio Economists from over 30 Ohio higher educational institutions conducted by Scioto Analysis. The goal of the Ohio Economic Experts Panel is to promote better policy outcomes by providing policymakers, policy influencers, and the public with the informed opinions of Ohio’s leading economists. Individual responses to all surveys can be found here.

What information do policymakers use to make decisions?

Last week, I was at the Association for Public Policy Analysis and Management’s annual research conference. In past years at this conference, I’ve spent most of my time at sessions talking about specific areas of policy research. This year however, I got to go to more sessions looking more generally about the tools of policy analysis. 

One of these sessions had a presentation by someone from The People Lab at Harvard. This was not necessarily a presentation about methods like some of the more technical presenters, but rather it was what information researchers include in their papers compared to what information policymakers use to select policies. 

To do this, the researchers are surveying as many local government employees as they possibly can. In the survey, government employees are shown two randomly generated policies and some key characteristics about them. For example, they might be shown two policies to address housing in a hypothetical city. Policy A has a large effect size but is high cost, Policy B has a smaller effect but a lower cost.

On each policy, the respondent is shown information about the experimental design (e.g. is it a randomized control trial or an observational study, etc), the policy’s general effects (e.g. effect size, variance, costs, etc), and some other contextual factors (e.g. political feasibility, would it require hiring new staff, etc).

Based on their preliminary results,* the researchers were able to determine which factors were most important in the decision making process for government officials. The five most important things that drove decision making were:

1. Short-term outcomes

2. Political context

3. Long-term outcomes

4. Study year

5. Whether a study had been replicated

The second part of this research project was to scrape data from research journals and see how often these different topics were brought up in academic research papers. Looking at the same list again, the reported percentage is the share of academic papers that include some mention of each of these categories. 

1. Short-term outcomes - 95%

2. Political context - 5% 

3. Long-term outcomes - 57%

4. Study year - 98%

5. Whether a study had been replicated - 9%

Compare this to the categories that policymakers cared the least about and we can clearly see that challenges that communicators of research need to overcome: 

     10. Sample size - 94%

     11. Statistical significance - 93%

     12. Evaluation method - 100%

Policymakers do not care as much about some of the things that researchers spend the most time thinking about. Those bottom three categories are three of the most important things I look for when trying to determine the relative quality of a research study. In an ideal world, policymakers would care more about these too since they are good indicators of the chances that a policy studied elsewhere will have similar results in their jurisdiction. 

However, I don’t just think that policymakers need to care more about variance. This list also clearly shows academics some ways they can make their research more relevant. Take the relative importance of long-term outcomes. Only a little of half the studies these researchers looked at measured long-term outcomes, despite the fact that it is something policymakers care a lot about. 

Additionally, policymakers care a lot about whether a study has been replicated. If an academic is only replicating a past paper, then they aren’t contributing nearly as much new information to the field. Replication studies are still extremely important for increasing our understanding of a topic, but there exists a bias against such studies.** 

The final and largest gap between academics and policymakers is the discussion of the political context. While it may not be appropriate for an academic paper to take a political stance, it would be beneficial to policymakers if there was some information about the political context in which a program was studied. 

There is a lot we can take away from these early survey results. We know this gap between academics and policymakers exists, and we at Scioto Analysis try to bridge that as best we can. Still, we’d be better off if policymakers and academics were more aligned on the things that matter most. 

* This was presented as a work in progress. The researchers were receiving their first wave of survey results the week leading up to this conference, and they expect these results to change somewhat by the end of the project.

** Replication studies that find different effects than the original are actually preferred, but if a researcher thought a study was well done and that they would find similar results then they would be disincentivized to try and replicate it.