Ohio Department of Education leading quiet policy analysis revolution

Last month, I attended the Association for Public Policy Analysis and Management’s Fall Research Conference. This is the ultimate wonk conference — more than 2,000 policy analysts and researchers convening in Atlanta, Georgia to talk about the most recent research on topics of public policy.

I go to this conference because I’m interested in what people are learning about public policy across the country. I’m often interested in learning things that I can bring back to Ohio — new analysis being conducted that is not happening here in the Buckeye State. Ohio isn’t usually on the cutting edge of policy analysis, so this is a good place for me to learn about things I can bring back home.

Imagine how surprised I was when I saw one of the most innovative research projects in the country presented by the Ohio Department of Education.

If you follow Statehouse news, you likely have heard about the efforts to reform education finance in Ohio. Alongside these legislative reforms, which will likely lead to billions of dollars in changes for school funding in Ohio, the state Department of Education (newly changed to the “Department of Education and Workforce”) has been conducting a series of studies on the cost of education in Ohio.

Two of these studies were released late last year. 

In November 2022, the Department of Education released a study by the American Institutes for Research on the cost of adequate special education in the state of Ohio. A month later, the Department released a study by West Ed and APA on the cost of education for English language learners in Ohio.

What I found fascinating about these studies was the approach they took. The studies were focused on a similar question: what will it cost to provide an adequate education for key student subgroups? They then answered these questions by turning to Ohioans.

Each of these studies included both interviews and surveys with professionals across the state to understand the components needed in education and the costs associated with these components. They both also undertook a “professional judgment panel” approach that utilized panels of local experts to understand the resources needed to provide education and the cost of those resources.

The Department of Education and Workforce is now contracting a new study, this time focusing on economic disadvantage, a component of school funding that could have a wider research than the last two studies.

While the Department has not officially endorsed the findings of these reports, they commissioned them in order to make sure that policymakers had access to the best information possible when formulating school funding policy.

Whether the General Assembly incorporates the results of these findings into future education budgeting is yet to be determined. We still live in a democracy, so it is not technocrats who make these decisions, it is elected officials who do. 

That being said, these sorts of studies represent a triumph for evidence-based policymaking and a marrying of the ideals of rational policy analysis and local input. Often Ohio is the last to undertake innovations in policy, but this is a situation where Ohio is leading the way. And as a state, we should be proud of that.

This commentary first appeared in the Ohio Capital Journal.

Effects of paid family leave policies

Paid family leave is one of the most commonly discussed topics among policy wonks in the US, which is why at the Association for Public Policy Analysis and Management conference there were six unique sessions on the topic. One of the reasons there is such a staggering amount of research on this topic is because access to leave after birth of a child or adoption can have such a wide range of outcomes for parents and children. 

In our research, we’ve almost exclusively focused on paid family leave as an anti-poverty policy. However, at this conference few researchers were focused on the poverty implications. Although I was slightly disappointed I couldn’t learn more about the impact that was most relevant to our work, this was a great opportunity to better understand some of the other most important outcomes paid family leave has.

Who actually takes leave

One interesting finding from multiple researchers was that paid leave was taken by high-income families more than low-income families. This was initially surprising to me because I assumed that these families with additional resources would choose to pay for child care and remain in the workforce to presumably increase their odds of career advancement. 

What I overlooked and what these researchers uncovered was the fact that paid leave almost never fully replaces an individual’s wage. A paid leave policy might cover 50% - 80% of someone’s salary for a few weeks. The result of this is that low income workers who are eligible for paid leave actually can’t afford to take it. High-income households have more capacity to absorb this short-term reduction to their income than low-income households. The partial loss of income is too severe for low-income mothers to maintain and they are forced to go back to work much sooner than upper-income mothers.

The gender wage gap

Another gap between intuition and reality, I expected paid leave policies to help shrink the gender wage gap, the logic being that women are better able to stay attached to the work force and earn higher wages as a result. In reality, multiple researchers found that family leave policies were associated with a stagnation in the gender wage gap. 

In short, the gender wage gap was shrinking during the 70’s and 80’s as more women entered the workforce, but this effect stalled once states began introducing family leave policies (these were largely unpaid leave policies). The researchers concluded that this stagnation was not because these policies could be thought of as benefits that only women received. While I don’t think the correct conclusion from this result is that paid leave policies are actually sexist, it is important to remember that paid leave isn’t a silver bullet when it comes to equity in the workplace.

Firm effects

One paper presented at the conference was focused on the way that firms responded to paid leave policies. Specifically in the context where firms were faced with a new payroll tax that the government would use to subsidize family leave when it was taken. 

They found that firms largely fell into three categories. First, large employers were essentially unaffected. For these companies with hundreds or even thousands of employees, neither the additional tax nor the temporary loss of an employee were particularly harmful. 

Second, small businesses were actually hurt by these policies. They were sometimes unable to deal with either the tax or the temporary loss of an employee. This is an important reminder that policymakers should be thinking of ways to balance out the potential downsides of policies like this. 

Finally, firms that have large percentages of female employees actually felt this policy as a subsidy. The payroll taxes were offset by the additional benefit their employees were receiving.

We all hope paid family leave can be a way to level the playing field for families. But effects can go different ways in the short-term. Good public policy will help ameliorate these problems to ensure that young children can spend time with parents, furthering their development and helping them escape cycles of intergenerational poverty.

When is cost-benefit analysis not enough?

Earlier this month, I was at the Association for Public Policy Analysis and Management conference, where I participated in a community discussion about economic evaluation of public policy. During this conversation, we talked about the differences between evaluating effectiveness, efficiency, and equity of public policies. 

It would be amazing if the most effective policies were also always the most efficient and equitable, but in reality we are faced with tradeoffs between these three goals. Do we prefer an efficient policy that only works on a small scale, or will policymakers stomach losing some efficiency to increase the magnitude of the result? What about a policy that is less effective overall, but does a lot more to improve equity–is this a tradeoff we would accept?

In cost-benefit analysis, one of the most important techniques for efficiency analysis, there is sometimes a desire to incorporate elements of equity analysis into the model. For example, if we include some measure of the diminishing marginal utility of income in a model, then whatever analysis we do will by default begin to suggest that more equitable outcomes (in this case those with less income inequality) are more efficient. Dan Acland, an economist from Berkely, has presented research on this topic at the annual Society for Benefit-Cost Analysis conference.

This begs the question, is it possible for cost-benefit analysis to go beyond efficiency analysis and also act as a tool for equity analysis?   

One of the main concerns with attempting to include criteria other than efficiency into cost-benefit analysis is the difficulty that comes with trying to monetize equity impacts. Monetization is often one of the most challenging parts of cost-benefit analysis, and the further we get from goods that are actually traded in a competitive market the more difficult it becomes to understand how much people are willing to pay for certain things. 

There are plenty of very rigorous ways to determine prices for things that are never actually bought or sold. Returning to the marginal utility of money example, economists have spent a lot of time researching and as a result we have a pretty good understanding of how valuable money is to different people. 

Instead, if we were talking about how much people would be willing to pay to avoid experiencing racism, we would have a much more difficult time coming up with a reasonable monetary value. Maybe you could connect it to people’s willingness to pay for things like home security somehow, but that would clearly be a weak connection. Monetization of impacts is a great strategy for estimating efficiency, but it is not necessarily the best way to measure effectiveness or equity.

More importantly, it probably isn’t appropriate to ask the question how much are people willing to pay to avoid experiencing racism. Cost-benefit analysis is really good at measuring efficiency, but there is no reason to expect it to be able to handle big questions of morality.

What does all of this mean for cost-benefit analysis? I believe that cost-benefit analysis should monetize equity outcomes when it makes sense while still recognizing that it is not a form of equity analysis. 

I think it makes sense to model the marginal utility of income in cost-benefit analysis because money can be used more or less efficiently by certain people. There are equity implications to that, but at its core the marginal utility of income is a question of efficiently allocating resources. However, saying income inequality is inefficient is not the same as believing we want to live in a society that distributes income more equitably. If we really want to care about the equity implications of policy decisions, then we need to dedicate time and energy beyond cost-benefit analysis.

Ohio economists agree land value tax will encourage property development

In a survey released this morning by Scioto Analysis, nine of twelve leading Ohio economists agreed that replacing property taxes with land value taxes would encourage property development. This is largely due to the fact that property owners would not see increased tax bills after making improvements to their property. In other words, there is no disincentive to additional property development. 

As Curtis Reynolds from Kent State wrote “Theory is quite clear that land taxes are more efficient than property taxes, so switching to land taxes should increase property development. How much that will happen is unclear.” 

Economists were split on the question of whether land value taxes are more progressive than property taxes. In general, property taxes are often regressive, meaning lower income individuals spend a larger percentage of their income on these taxes than higher income individuals. This is because lower income individuals generally spend a greater percentage of their income on housing than higher income individuals.

As Jonathan Andreas from Bluffton University wrote: “The devil is in the details and I don't have all the details, but for urban taxes it seems hard to engineer a land value tax to be less progressive than a real estate tax because poorer people spend a larger percent of their income on housing (which is taxed less) whereas richer people own most of the land (which is taxed more).”

The Ohio Economic Experts Panel is a panel of over 40 Ohio Economists from over 30 Ohio higher educational institutions conducted by Scioto Analysis. The goal of the Ohio Economic Experts Panel is to promote better policy outcomes by providing policymakers, policy influencers, and the public with the informed opinions of Ohio’s leading economists. Individual responses to all surveys can be found here. 

How to adjust for inflation in social return on investment analysis

The biggest economic story of the past year has no doubt been inflation. As the world has dug itself out of the COVID-19 recession, supply chain gluts and reduced labor supply has driven prices up across the world.

In the United States, inflation peaked at 9.1% in June of 2022, over quadruple the Federal Reserve’s goal of 2% inflation. Inflation has since cooled, dropping to 3% a year later in June of 2023 and staying in that area into the fall of 2023.

This nationwide price whiplash is likely to have an impact on practitioners of cost-benefit analysis and social return on investment analysis. Recently, in some of our work with Ohio University’s social return on investment analysis team, we had some conversation about the impact of inflation on valuation.

Inflation means dollar values can buy different bundles of goods over time. In general, it means that more dollars are needed to purchase the same goods or services as time goes on.

This has an impact on analysis that relies on monetization to compare goods and outcomes against one another. If one study monetizes the value of an outcome in 1990 at $100 and another study monetizes the value of a separate outcome in 2022 at $100, those two outcomes are not the same. The two values need to be brought to the same year value in order to be compared to one another.

Guides to social return on investment do not typically provide a lot of guidance to practitioners on how to incorporate inflation into their analysis. Social Value International’s Guide to SROI does not mention “inflation” throughout the entire document.

We were able to find one guidance document on social return on investment analysis that mentions inflation. New South Wales, Australia’s Department of Communities and Justice Guide to SROI recommends analysts “adjust dated historic values for cost inflation so that they can be compared to contemporary values.”

In response to this dearth of guidance on the use of inflation in social return on investment analysis, we have provided some tips for the practice.

1. Consider inflation when doing valuation

Whenever a number is reported, identify the year the data was collected. Adjust numbers to bring them in line with the year the social return on investment is being calculated for. This allows apples-to-apples comparisons to be made across time when conducting analysis.

1. Use the Bureau of Labor Statistics’s Inflation Calculator

The Bureau of Labor Statistics Inflation Calculator is an excellent, reputable tool for estimating the impact of inflation on average prices. To use the tool, set the first number and date based on the source and the final date based on the date the study period applies to.

1. Include both the original value and inflation-adjusted value in your SROI Model

Transparency is clear when modeling quantitative and monetized impacts of programs in a social return on investment analysis. Include both the original value from the study as well as the inflation-adjustment value in your documentation so people can see how the number was changed using the inflation adjustment.

While this adds another step, it makes the number transparent and easier to verify during data checking. It also makes clear that inflation adjustment happens, helping you to reduce the risk of accidentally doubling the inflation adjustment and making it clear to those who verify the analysis that inflation adjustment took place.

1. Use alternate inflation measures when appropriate

Some sectors of the economy grow faster than others. If a valuation refers to a sector that has grown faster or slower than the general economy, consider using a producer price index based on that specific industry rather than the consumer price index to more accurately reflect the changes in prices over time. This can make the analysis fall closer to the actual value when a market is being considered that has anomalous inflation over time.

These are just a few tips that can make your use of inflation in social return on investment more rigorous and bring your analysis close to the truth about the value of programs analyzed.

What happens if legislators rewrite Issue 2?

Earlier this month, Ohio became the 24th state to legalize the sale and purchase of marijuana for recreational use. Even before Issue 2 was passed by the voters, leaders in the Ohio General Assembly were signaling their intentions to change the law after its passage.

Unlike Issue 1, which was a constitutional amendment, Issue 2 is an initiated statute.

That means legislators can go back and change the law afterwards. And it seems like they intend to do so.

While some of the public discussion about changes to Issue 2 from the legislature have revolved around public health measures, legislative leaders have focused their attention on the use of tax revenue raised by excise taxes levied on purchase and sale of marijuana for recreational purposes.

Researchers at the Ohio State University Drug Enforcement and Policy Center study estimate hundreds of millions of dollars are likely to be raised by the excise tax.

My firm Scioto Analysis decided last summer to conduct a cost-benefit analysis of recreational marijuana legalization. This lined up perfectly with the discussion around Issue 2. We released the cost-benefit analysis last month, finding the current proposal would likely lead to economic benefits exceeding costs by $260 million.

Legalization of the sale and purchase of marijuana for recreational purposes in Ohio will not be costless. We expect there to be new cases of impaired driving that will likely lead to crashes and even deaths. But the largest impact we expect is productivity. Based on what we have seen in other states that have legalized recreational marijuana, we expect the costs of recreational marijuana to rise to the level of $760 million worth of lost productivity.

These costs, though, will be offset by the benefits of programs funded by excise taxes levied on the industry. In particular, if the Cannabis Social Equity and Jobs Fund and Substance Abuse Addiction Fund authorized by Issue 2 are spent on evidence-based programs, we estimate its benefits to be about $800 million in job training and substance abuse treatment benefits.

Changing the use of these funds, though, could potentially have impacts on the economic efficiency of recreational marijuana legalization.

A big problem with reallocating these funds toward policing from an economic standpoint is that policing is an incredibly understudied public policy area. Very little work has been done to evaluate the economic impacts of policing interventions and less has been done on impacting the economic benefits and costs of investing in jails and prisons as some assembly leaders have suggested.

On the other hand, the current evidence on job training and substance abuse programs
suggests benefits on the scale of $6 and $9 respectively for every $1 in costs.

The best evidence from the Washington State Institute for Public Policy suggests deployment of police can have benefits that outweigh costs. Deploying police can lead to reductions in property crime, about to the tune of $5 in benefits for every $1 in costs.

While this is one way to tackle crime, another is to fund general job training programs, which reduces the value of crime relative to regular employment activity.

Training is a different story. The evidence for the economic benefits of new police training is not strong right now. This leads me to conclude that reallocating money currently allocated toward job training and substance abuse treatment toward police and jail construction will likely reduce the economic efficiency of marijuana legalization.

Let us hope that if we are going to reallocate funds going toward effective programs that we will at least put resources toward evaluating these new programs they fund.

This commentary first appeared in the Ohio Capital Journal.

The evolving story about marijuana and public health

Last month, Scioto Analysis released a cost-benefit analysis looking at the recreational marijuana ballot initiative that passed on election day. Our goal with this analysis was to hopefully offer some insight into what the impacts of legalization would be so Ohio voters could make an informed decision. 

Overall, we found that the ballot initiative as written would likely generate over $250 million in social benefits for the state. One notable exclusion from our analysis was the lack of a monetized health impact. In other words, we did not explore what increased marijuana use would mean for public health. 

There are a few reasons we chose to exclude this particular impact. The most important reason was the uncertainty that surrounded it. Specifically, we were unsure of the effect that recreational marijuana legalization would have on alcohol use. Some studies have found that states with recreational marijuana use see decreases in their alcohol consumption, and if that is true then we might expect some of the negative health outcomes associated with marijuana use to be balanced out by positive health outcomes associated with less alcohol use.

Evidence of the negative impact of recreational marijuana legalization on health was presented last week at the Association for Public Policy Analysis and Management conference during a session about the impacts of recreational marijuana laws. One presenter in particular was able to measure the impact that recreational marijuana legalization had on asthma hospitalizations.  

Taking this new information into account, our final results probably would not change substantially. Doing some back-of-the-envelope math, I figure these added asthma cases would have a monetized value somewhere in the tens of millions. Certainly a major cost and an important consideration, but not enough to change a margin in the hundreds of millions.

Another consideration is the fact that this research was being presented as a work in progress. The researchers’ estimates were preliminary, with more questions that need to be addressed in order to fully understand the connection between recreational marijuana legalization and asthma hospitalizations. 

This situation highlights the differences between academic research and policy analysis. Academics need lots of time to carefully estimate each individual effect, while policy analysts need to provide useful information quickly. The goal of academic research is to poke holes and see how solid an argument can be made. This is with the goal of advancing “knowledge” writ large. Policy analysis, on the other hand, is focused on improving the knowledge policymakers have when they make decisions. This is usually on a shorter timeline than academic publishing.

As more states legalize recreational marijuana, there will be more high quality research about the associated health impacts of the legalization of the sale and purchase of marijuana for recreational purposes. I especially hope some academics are up to the challenge of measuring net health benefits, including any impact these laws have on alcohol use.

Either way, according to the evidence we have now, Ohio’s economy will be better off from a social perspective as a result of this vote (assuming the legislature doesn’t reallocate the tax dollars currently allocated by the program for employment and drug abuse treatment). In the event there is groundbreaking research about the negative health effects of marijuana use, then most of the country is going to have to reevaluate their laws. This is the purpose of the laboratories of democracy, though: to test policies and see what happens when policy changes.

Will more school spending improve student achievement?

The Journal of Policy Analysis and Management has a section at the end called “Point/Counterpoint” where two people present different sides of an open question in the field. In the most recent edition, the subject of the Point/Counterpoint was school spending. 

Scioto Analysis is currently conducting a cost-benefit analysis on school spending in Ohio, so this is a great place to explore the strengths and weaknesses of our approach. So, let’s go over some of the key points the researchers make.

Before going over both sides, it’s important to understand exactly what is at the heart of this debate. Neither side believes that school spending is unrelated to student achievement, instead the question is whether we can use spending broadly as a predictive variable. In other words, we are trying to figure out how much per-student spending alone can tell us. 

To illustrate this, let us begin by looking at the negative side.

Counterpoint

This section written by Josh McGee from the University of Arkansas is titled “Yes, money matters, but the details can make all the difference.” The thesis of this argument is that knowing the school spending amount alone is not enough, we need to have an idea of what these additional resources actually are.

From an intuitive standpoint, this claim makes a lot of sense. Not all spending is worth the same amount and there can be a lot of value in finding the best way to spend additional resources. 

For policy analysis, this would suggest that using school spending alone as an input into a model would not be enough to accurately estimate what the impact of a proposal would be. We'd need to know more about what that money would actually be going towards. 

One final point McGee makes is that school spending might be extremely context dependent. It might not make sense to use the results of increasing school spending in one context to inform policy somewhere else. 

Point

The argument supporting the use of school spending  as a predictor of student achievement written by C. Kirabo Jackson from Northwestern University and Claudia Persico from American University is fairly straightforward: so far, the data suggest that there is a causal connection between spending and achievement. 

A recent meta-analysis of the literature about the impact of school spending found that there is a positive relationship between overall per student spending and test scores. This trend has been consistent over time and across multiple contexts. 

If we assume this is true, then indeed we could use generic school spending as an input into some model. More information about exactly how this spending would be used could certainly be helpful still, but overall there is a measurable effect of school spending.

—

What does this mean for policy analysts looking to measure the impact of increased school spending? Mostly that there is still some uncertainty about exactly what the value of increased spending exactly is. 

It is good that academics challenge these results, and work to see if there is a way we can improve our understanding of this topic. However, I believe that the results showing a direct link between spending and student achievement are strong enough to improve decision making, even if they aren’t perfect. For policymakers and policy analysts, this helps improve the quality of our decision making. 

Ohio economists believe medical debt forgiveness will improve health

In a survey released this morning by Scioto Analysis, a majority of economists agreed that using public funds to forgive medical debt will reduce health disparities. This comes just after the City of Columbus voted on a plan to forgive medical debt for middle income residents. 

Paul Holmes from Ashland University who agreed that forgiving debt would reduce health disparities wrote “I suspect this would reduce the number of people avoiding medical care for financial reasons, though again I don't expect the effect to be large. But it does seem relatively well-targeted, so I'd be optimistic it might make health care access more equitable.” 

Kathryn Wilson from Kent State who was uncertain about this proposition wrote “I think it will definitely reduce economic disparities. Those who have their debt purchased will have more disposable income, a higher credit score, and be in a better position to finance purchases such as a house or car in the future. The economic security may provide some health benefits, but I see the primary effect as reducing economic disparities rather than health disparities.”

A plurality of respondents agreed that this policy would additionally encourage people to access preventative care, though more were skeptical of this claim. Bob Gitter from Ohio Wesleyan wrote “I am not sure that preventative care is a big share of medical debt. Also, this program forgives debt for past expenditures. It is hard to know if an expenditure I make now would get forgiven later.”

The Ohio Economic Experts Panel is a panel of over 40 Ohio Economists from over 30 Ohio higher educational institutions conducted by Scioto Analysis. The goal of the Ohio Economic Experts Panel is to promote better policy outcomes by providing policymakers, policy influencers, and the public with the informed opinions of Ohio’s leading economists. Individual responses to all surveys can be found here. 

What will Ohio look like if Issue 1 fails?

Next week, Ohio voters will go to the polls to vote to establish a state constitutional right for residents to make reproductive decisions about abortion, contraception, and miscarriage care.

In the wake of the decision made by the U.S. Supreme Court in Dobbs v. Jackson Women’s Health Organization to overturn the court’s previous opinion of a federal constitutional right to abortion, state legislatures across the country have moved to ban or severely limit abortion.

Ohio currently borders three states with total bans on abortion: Indiana, Kentucky, and West Virginia. Abortion is currently legal in Ohio due to a judicial order to freeze a 2019 law that would ban abortion at around six weeks of pregnancy.

A group of economists led by Middlebury College’s Caitlin Myers have studied the impact bans have and will have on access to abortion in the United States.

Currently, the average drive time to an abortion facility in Ohio is around 36 minutes. This estimate is nearly identical to the 34 minutes researchers at health research firm Altarum estimated Americans typically drive to the doctor’s office.

Because of clinics in Akron, Cincinnati, Cleveland, Columbus, Dayton, and Toledo, people living in those metropolitan areas can generally reach a clinic that provides abortion in under an hour.

Proximity to those clinics as well as clinics in Pittsburgh mean most of the rest of the state is currently within a two hour drive of a clinic that provides abortion. The exception is eastern Lawrence County on the Kentucky and West Virginia borders, where bans in neighboring states and distance from cities in Ohio and Pennsylvania push the drive time over two hours.

The researchers estimate that if Ohio’s abortion ban went into effect, average drive time in Ohio would grow from the current 36 minutes to over two and a half hours.

People living in the Cincinnati, Columbus, and Dayton metropolitan areas and large swaths of the Cleveland Metropolitan Area would have to drive over two hours to reach a clinic that provides abortion. People in Akron, Toledo, Youngstown, and portions of Cleveland would still have access to clinics that provide abortion within an hour in Pittsburgh and southern Michigan.

Portions of southern Ohio would be over four hours from a clinic.

The researchers also estimate the current average distance to a clinic that provides abortion in Ohio is 28 miles but that the ban would increase that distance to 159 miles. A 2020 study by a group of researchers at Texas A&M found increasing the distance to a clinic from under 50 miles to over 50 miles reduces abortion rates by 16 percent, suggesting distance will have an impact on how people access abortion care in Ohio. This would likely lead to an increase in rates of unplanned births and self-managed abortions.

If voters in Ohio pass Issue 1 next week, the right to abortion will be ensured up to the point of fetal viability, which would ensure the right to choose abortion care is limited to women and their doctors. We will find out next week if voters in the state are ready to codify a right the Supreme Court was not able to.

This commentary first appeared in the Ohio Capital Journal.