How to determine thresholds in policy analysis

For the past year, Scioto Analysis has been working with the Center for Climate integrity on a series of projects that attempt to measure the financial impact climate change will have on local governments. The first report on Ohio was released in July of last year, and just last month the newest report on Pennsylvania came out. 

In total, the report projects that Pennsylvania’s local governments will have to spend over $15 billion by 2040 in order to adapt to climate change. That’s $15 billion to roughly maintain the same living conditions we have today in the face of climate change.

The report goes into a ton of detail about these costs, and there is even an excellent interactive tool our partners on this project, Resilient Analytics, put together in case you want to learn more about this project.

However, I wanted to dive deeper into one aspect of this report. How we defined the categories for our equity analysis. 

Briefly, the equity analysis for this project involved identifying certain criteria that municipalities had to meet in order to get labeled as a certain equity category. For example, a municipality that has a poverty rate over 20% was labeled as high poverty. 

Conceptually, we are trying to identify municipalities that are more vulnerable to climate change, that have less adaptive capacity, and that have been historically marginalized. The difficulty comes from the fact that we need to determine a specific cutoff point.

This problem comes up across all sorts of contexts. Think about how small the difference in wellbeing is for one person who earns $1 above the poverty threshold and a person who earns $1 below the poverty threshold. Small differences in experience but entirely different categories. 

In most cases, we can turn to the standards set by others who have done research on this before. If there is a consensus among the community that studies these topics, then it is usually not our place to come to a different conclusion. Having consistency with past research on the same topic not only helps tie our work to the established literature, it also makes it easier to communicate the final results.

One way we defined our thresholds when there was not outside guidance was to try and make the new groups comparable to those we had already defined. In the case of our Pennsylvania analysis, we did this by making sure that roughly the same number of municipalities fell into each equity category. 

This approach also ensured that we were getting enough of a sample size in each category to make reasonable inferences. Making sure that one group isn’t too small is very important to this type of analysis. 

Whatever thresholds we choose to use, we need to make sure that they are well defined before we begin our analysis. It might be tempting to wait until the analysis is done and see what thresholds provide interesting results, but that is answering a different question. It would be dishonest to determine the thresholds after the fact and report comparisons between the two groups.

What to do with conflicting information

As policy analysts, we are often at the mercy of the research others have done in order to estimate some outcome. Rarely in this job do we have the time to sit down and research a problem the same way an academic does. Instead, we focus on finding creative ways to take the research that others do and apply it to the context that we are interested in understanding.

Unfortunately, academic research doesn’t always agree with itself. 

Sometimes it’s a small difference, and the final results don’t really change all that much. But sometimes other research is directly contradictory, and our outcome will depend heavily on what estimate we choose to use.

In the case of a small difference, we could do something like use the average effect and wait until sensitivity analysis to explore the full range of outcomes. In the case where there are contradictory results, we need to make a decision. 

Anytime we make a decision like this, we need to be careful to fully understand what the implications are and communicate them effectively. Here are a few things I try to think about when I am presented with conflicting information. 

What context is most similar?

The first place I always begin when thinking about what research I want to use in my own projects is how similar is the context of my situation. It is more reasonable to think that the estimates others have come up with will more closely hold if we change fewer things about the situation in which we are applying them. 

For example, when working on a project estimating the cost of climate change in Pennsylvania, it would be much easier to use climate research from places with similar climates today. Ideally, we’d want studies that look at climate change in Pennsylvania specifically, but if someone measured the cost of climate change in Ohio that would still be a useful piece of research. 

Conversely, it would probably be incorrect to use estimates for the cost of climate change measured in Brazil. Even though those researchers might have come up with a very detailed causal equation that neatly ties increased temperatures to monetary losses, we know for a fact that the underlying assumptions about the climate are different there than in Pennsylvania.

Which estimate is the easiest to translate?

Another important consideration is how much work is it going to take to manipulate someone else's results and make them usable. I don’t necessarily mean how much effort it takes (though time is a limited resource and should always be considered) but rather how many steps and assumptions are required to use someone else’s result.

Each time we inflate a value, adjust for regional differences in prices, or change units, we introduce opportunities for estimates to become less meaningful. Some of these are more important than others, changing from pounds to kilograms for instance should not interfere with anything. 

—

These are just two things to consider when deciding what estimate to include in a study. Ideally, during the sensitivity analysis phase we can explore and report how these different estimates would impact the results. 

How can we use policy to reduce disparities in Ohio?

Earlier this month, the Health Policy Institute of Ohio released a brief on the prevalence of racial disparities in Ohio. The study quantified the disparity between racial groups of Ohio at $79 billion from gaps in income, consumer spending, tax revenues, health care spending, productivity, and corrections spending.

Racial inequality has economic implications, but it’s also something worth eliminating for its own sake. But how do we do this?

The brief from the Health Policy Institute of Ohio has a few suggestions. It suggests implementing policies that promote justice and fairness, tailoring policies to support people of color, allocating resources to community-building policies, evaluating disparities, reforming criminal justice, and ensuring equitable access to financing.

I’ll add some other policies to consider.

Income

One of the biggest disparities between different racial groups in Ohio is income. According to 2021 5-year American Community Survey data, Black Ohioans are 2.6 times more likely to be in poverty than white Ohioans.

Income disparities could be closed through tax policy. One option is the state earned income tax credit, a tax credit that goes to low-income workers. Ohio’s earned income tax credit rate is rather high, but because taxpayers can’t receive any more from the credit than they pay in taxes, its ability to alleviate poverty is hampered compared to the federal credit. Changing this policy would put more dollars in the pockets of low-income workers, which would help reduce the racial income gap.

Another option is the child tax credit. This credit gives cash assistance to families with children. Recent analysis by my practice revealed that creating a state child tax credit in Ohio could generate anywhere from $60 to $300 million in net economic benefits, mostly realized through higher future incomes for children in families that receive the credit. Adopting a state child tax credit could help narrow the racial income gap by helping low-income households with chilren.

Housing

Housing is a living cost no one can avoid. A National Association of Retailers study from earlier this year found 30% of Black homeowners in the United States are housing cost burdened, meaning they spend over 30% of their income on housing. This compares to only 21% of white homeowners. If housing costs can be controlled, it could have an impact on closing the gap between Black and white households.

One way housing costs can be controlled is by encouraging construction of housing. Reducing restrictions on construction of multifamily housing such as those promoted through single-family zoning could ease the cost of housing, especially in more tight housing markets.

Education

A 2020 brief from the National Institute for Early Education Research found black children are nine months behind white children in math achievement and seven months behind white children in reading achievement as early as kindergarten entry. Promoting access to early childhood education can be a tool for leveling the playing field between people from different racial backgrounds.

We have tools for reducing disparities between different groups. Reducing these disparities will help Ohio’s economy, and it is also just the right thing to do. Ohio should not be a place where people’s outcomes are predetermined at birth based on skin color or ethnic background. It should be a place where the field is level for people of all different backgrounds to contribute. Policy can be a tool for making that a reality.

This commentary first appeared in the Ohio Capital Journal.

Is medical debt forgiveness good public policy?

Starting Scioto Analysis was the first time in my life I was living without health insurance. One of the things I felt like I could trim at the time was visits to the dentist: it felt like a luxury to get regular checkups and I brush and floss regularly, so I felt like I could get away with it.

Boy was I wrong. When I went to the dentist last year for the first time in four years, I had gum problems that develop when you don’t go in for regular cleanings. The dentist put me on a new mouthwash regimen and had me coming in for regular cleanings. I ended up racking up over $1,000 in dental bills for the year of treatment.

Last week, I was in the office for my regular cleaning. My gums had shown considerable resilience and were almost where they needed to be, but I still had some problem areas. The dentist recommended a targeted antibiotic regimen. This would cost another $600.

This is a lot to fall onto someone’s plate, but I’m not alone in the amount of out of pocket medical expenses I incur. And I have the means to pay for them. This isn’t the case for everyone.

A 2020 survey suggested Americans’ collective medical debt was nearly $200 billion in 2019. Nearly 5% of Americans owe over $2,000 in medical expenses. 1% of Americans are in deep medical debt, owing over $10,000 in medical expenses.

These expenses can have a significant impact on family incomes. A Census Bureau analysis found medical out of pocket expenses pushed 5 million people into poverty in 2020. Families that are cash constrained often pass on paying off medical debt in order to pay for other essentials such as food and housing.

Some solutions have emerged to reduce medical debt for struggling people. RIP Medical Debt is a nonprofit that partners with communities to assess where medical debt is least likely to be paid, purchase it from collectors for pennies on the dollar, then forgive it.

The Columbus, Ohio City Council is currently considering a proposal to partner with RIP Medical Debt to cancel millions of dollars of debt for Columbus residents. Debt cancellation would be targeted to low- and middle-income families with debt owed to Columbus’s four main hospital systems.

Debt cancellation can have impacts on health utilization. A 2021 study of a debt forgiveness program for low-income patients at Kaiser Permanente hospitals in Northern California found debt forgiveness led to a sharp increase in visits to the doctor among those who received debt forgiveness. This led to higher detection of serious medical conditions like heart disease and diabetes and a large uptick in prescription refills for treatment of high cholesterol, diabetes and depression.
Medical debt cancellation isn’t a silver bullet for solving poverty, but it is one tool in our arsenal for helping a subset of the population who have become victims of the United States’s notoriously inefficient health care system. Franklin County Ohio’s Rise Together Innovation Institute is leading discussions on medical debt cancellation because of its promise to alleviate the burden of medical debt in the region. We’re looking forward to seeing what progress they can make on this front to help alleviate the burden for people who are struggling.

How to construct policy alternatives

Earlier this month, Scioto Analysis released our most recent example cost-benefit analysis looking at what a child tax credit might look like in Ohio. Our goal with releasing these analyses is to demonstrate what detailed policy analysis looks like in Ohio and to encourage more policymakers to make evidence-based decisions. 

In the spirit of helping demonstrate what the process of performing a well defined cost-benefit analysis looks like, I wanted to explain some of the key considerations when choosing policy alternatives and show how we apply them in our research. 

Choose the right number of policies

One thing to realize right away is that just like in policymaking, there are significant tradeoffs we should be aware of in the policy analysis process. The more policy alternatives we choose to measure, the more accurate our understanding of a particular policy will be. However, this comes at the cost of time and resources that could otherwise be spent on other more useful endeavors. 

Only choose alternatives that add sufficient information about the policy in question. In our case, we could have specified countless child tax credits, incrementing the values by pennies at a time. Of course, this doesn’t add enough additional information compared to picking three amounts for the credit. If you have the capacity and are interested in how pennies change policies, then an interactive tool that allows for very small marginal adjustments could be one way to see that without a ton of additional work.

Compare to relevant policies

Unless you are dealing with a truly novel policy proposal, it is often good practice to start with similar policies as the basis for your alternatives. Not only does this give you a clear place to look for data, but also it ensures that the alternatives you propose are politically feasible in at least some contexts. 

In our analysis, we based the amounts of the tax credits on what other states already have in place. This ensures that the amount of the tax credit would be reasonable for Ohio to implement and helps the policy makers focus on the most realistic options. 

Make adjustments that are relevant

The goal of measuring multiple policy proposals is to better understand the full scope of what a policy looks like in practice. We are interested in what some child tax credit might look like rather than what a specific child tax credit might look like.

Keeping the main structure of a policy intact and adjusting around the margins can identify what the range of outcomes is for policymakers and give them a sense of what levers there are to pull. 

—

The goal of defining good policy alternatives is to show policymakers what a range of reasonable specifications might look like for a given proposal. As policy analysts, it is our job to make sure that we choose our alternatives carefully and make sure we are getting as much information for the work we put in as possible. 

Hopefully the demand for quality policy analysis increases in Ohio in the future. Although policy analysis will always carry some amount of uncertainty, through careful research we have the ability to increase the quality of our decisions as a society should we choose. 

Ohio economists split on affirmative action decision

This morning, Scioto Analysis released a survey of Ohio economists exploring the impact of the Supreme Court’s recent decision to end affirmative action on Ohio’s colleges and universities. The plurality of respondents believe that this decision will reduce diversity among college students in Ohio.

Many respondents point out that the effect of this decision might be small and isolated to only a few select schools. “There is research based on states that removed affirmative action previously. That research generally shows that on average there was not much change in college attendance except at the most selective schools, where minority enrollment decreased and White enrollment increased,” wrote Curtis Reynolds from Kent State. “So it could decrease diversity at the most selective schools, but will likely not have much effect at most institutions.”

There was less consensus on the question of how this decision would affect the ability of colleges and universities to promote economic mobility. As many respondents pointed out, these effects have historically been isolated to historically selective schools. If enrollment doesn’t change much in places like large state schools, then the economic mobility provided by a college degree may not be affected very much. 

The Ohio Economic Experts Panel is a panel of over 40 Ohio Economists from over 30 Ohio higher educational institutions conducted by Scioto Analysis. The goal of the Ohio Economic Experts Panel is to promote better policy outcomes by providing policymakers, policy influencers, and the public with the informed opinions of Ohio’s leading economists.

Who benefits from a child tax credit?

Earlier this week, Scioto released a new cost-benefit analysis looking at what the impacts of a state child tax credit might be for Ohio. We chose to estimate the impacts of a state child tax credit because the federal child tax credit that was introduced as part of the American Rescue Plan Act went away last year. 

In its short time, that child tax credit was one of the most effective anti-poverty programs in decades, lowering child poverty to its lowest percentage ever recorded. Because it no longer exists at the federal level, we were interested in seeing how it could be implemented at the state level. 

Our analysis looked at three different plans for administering a child tax credit that varied based on the age of children eligible for the credit and the size of the credit. In all three of the plans, we found that economic benefits outweigh costs of the program. 

In addition to measuring the society-wide costs and benefits, we were also able to perform a distributional analysis to see exactly where these costs and benefits landed. 

Unsurprisingly, the majority of the benefits were received by the recipients of the tax credit and the majority of the costs were borne by those who don’t qualify for the tax credit. An important result of this distributional analysis is that if we only look narrowly at the impacts on households that don’t receive the tax credit, we find that there are actually slightly negative net benefits.

This is not to say that the people who don’t receive the tax credit don’t see any benefits from this program. Things like the expected reduction in future crime benefit everyone in society as taxpayers and possible victims of crime. However, for the people who would not qualify for this credit, this probably isn’t the most efficient way to achieve those same benefits.

For the people who qualify for this tax credit, the benefits of the program are enormous. The most significant benefit is the expected increase in future earnings for children who grow up with this extra income. For a poverty program, the ability to keep people out of poverty in the future is extremely important, and this intervention achieves that goal effectively and efficiently.

Generally speaking, this program is a small loss for people who don’t qualify, and a much larger gain for those who do. Because the qualification criteria is based on income, we know that this program benefits those who are less well off and does not benefit people at higher incomes. 

These insights should hopefully help policymakers understand exactly what tradeoffs come with this sort of policy. Yes, some upper-income households will have less resources and less ability to get the things they want. However, this is overwhelmingly offset by massive gains for those in our society who are struggling the most. 

Our analysis also highlights the fact that the majority of the benefits for this policy are realized in the long-term. In the short term, this helps people who are currently in poverty get a little extra income. As an investment, we are helping today’s children stay out of poverty as adults, and reducing the future burden on our social safety net. Even though there are tradeoffs, once we add everything up we expect this policy to make our society better on net.

Ohio child tax credit would boost state economy

This morning, Scioto Analysis released a new cost-benefit analysis estimating the impact of a state child tax credit for Ohio. We find that depending on the size of the credit, the state could generate between $60 million and $300 million in net benefits.

A child tax credit is a program that provides cash support for families with children. In addition to the federal child tax credit, 12 states have child tax credits of varying levels and availability, with credit amounts ranging from $100 to $1,000. A November 2022 Census Bureau analysis found that the federal child tax credit expansion in 2021 lifted 2.1 million children out of poverty.

“The expansion of the federal child tax credit in 2021 was the most significant antipoverty policy change in the United States since Lyndon B. Johnson’s Great Society,” said Scioto Analysis Principal Rob Moore.

According to the analysis, the majority of the benefits of a state child tax credit would be realized as increases in future earnings for children who qualify for the tax credit today. This means that in addition to lifting children out of poverty today, a child tax credit could help disrupt the cycle of intergenerational poverty.

Additionally, there are major social benefits in the form of reductions in future crime and healthcare expenses. 

“Programs like this that reduce child poverty are not just band-aid fixes that help people get by today, they are investments in the future that have major positive impacts for society” said Michael Hartnett, policy analyst. “Everyone in our society, even those who never see a dollar of this tax credit stand to receive major benefits.”

This study is the most recent cost-benefit analysis conducted by Scioto Analysis. Previous cost-benefit analyses include research on water quality programs, municipal tree planting, volunteer programs, and school closures for COVID-19.

Is paid family leave an anti-poverty program?

The first months of a child’s life are crucial for her development. Children who have more individualized attention from a caregiver have more of a chance of succeeding down the road. Encouraging parents to spend time taking care of infants could pay dividends for society down the road.

Throughout much of the United States, though, it is hard for parents with careers to do this. According to the New York Times, the United States is one of eight countries in the world with no guarantee of parental leave. The average country guarantees 29 weeks of parental leave and nearly all of Europe plus countries like Iran and Russia guarantee at least 24.

Among U.S. states, paid leave is also rare. Only seven states, five in the northeast and two on the west coast, had implemented paid family and medical leave laws as of June of last year according to the National Conference of State Legislatures. Four additional states had enacted legislation but not yet implemented it.

Despite only a smattering of states enacting paid leave guarantees, local governments across the country are considering paid leave as a way to improve working conditions and bolster prospects for children and youth. Because of this dynamic, Scioto analysis is working with the Rise Together Innovation Institute to explore opportunities for expanded parental leave in Franklin County, Ohio.

Lack of parental leave policies put families on the edge of poverty in a difficult position. They have to decide to either have children and not spend time with them in some of their most crucial developmental moments, have children and give up income to spend time with them, or postpone or forgo having children at all.

Putting a robust parental leave program in place can be an effective tool for empowering these families on the edge of poverty. First, they ensure families have resources when they have children and can spend time with those children during key developmental months. Internationally, more extensive parental leave policies correlate with a decreased risk of poverty for both two-parent households and single mothers.

As hinted at the start of this blog post, paid family leave is also an intergenerational policy. Early interactions between parents and infants have a large impact on long-term outcomes for children when it comes to cognitive and social development. Children who enter school and then the workforce with better cognitive and social skills have a leg up in escaping poverty in adulthood. By promoting time between parents and children at the earliest ages, parental leave is a multigenerational anti-poverty program.

Lastly, paid leave laws could be an effective tool for closing the gender wage gap. Since women are more likely than men to take leave, employers could be reducing wages for women to make up for the paid leave they currently provide. Requiring paid leave for all parents could theoretically combat this race to the bottom.

In May, Mayor Justin Bibb of Cleveland proposed a parental leave program that would apply to all city employees with a month of service, giving them 500 hours of parental leave at 100% of salary (20 hours are reserved for prenatal appointments/preadoption appointments). This is more robust than either the City of Columbus or Franklin County’s current policies.

Enacting new paid leave laws could be an effective tool for supporting parents now and children over the trajectory of their lives. And the policy is not only a workforce policy, but also an anti-poverty policy.

What is an Accessory Dwelling Unit (ADU)?

Recently Scioto Analysis has been working with the RISE Together Innovation Institute to research the current state of poverty in Franklin County and policy to improve it. One policy we’ve been looking at has been loosening regulations surrounding Accessory Dwelling Units (ADUs) as a way to improve housing affordability. 

One major challenge faced by people in poverty is finding affordable housing. We found during our research that families with income under $20,000 annually were 28 times more likely to be housing burdened (spending over 30% of their income on housing) as families making over $70,000 annually. 

In Columbus, housing prices have risen by over 60% in the last five years. To make matters worse, the Mid Ohio Regional Planning Commission projects that in 25 years, the Central Ohio region will grow by as many as 756,000 people. 

If nothing changes, Franklin County could be on track for a housing crisis.

The good news is that rising prices and a lack of housing have the same solution: increase the supply of housing. 

This can be achieved by building new affordable housing, but that takes a lot of time and resources. Even worse, developers are often incentivized to build new luxury housing in order to maximize the value of their property. 

This is where ADUs come in. An ADU is a separate living space that exists on a lot in addition to a single family home. It could be part of a single family home like a basement or an attic, or it could be a separate structure like a detached garage. 

In addition to increasing the supply of housing, ADUs also make neighborhoods with single family homes more affordable to live in. Without dramatically changing the existing infrastructure, we can put more families in neighborhoods that historically have been limited in the number of people that can physically live there. 

Another side effect of ADUs is that it gives low-to-middle income earners more access to neighborhoods with higher income. Those neighborhoods have higher upward mobility thanks to things like high quality schools and low crime rates.

Arguably the best benefit of ADUs is that from a policy perspective, they are extremely cost effective. While ADUs are not going to solve the problem of a massively growing population (that will require large investments into affordable housing), ADUs are a low-cost strategy to get more people into stable housing situations. All it requires from policymakers is a change in zoning rules. 

The benefits of having stable housing are significant as well. People with reliable housing have better health, employment, and education outcomes. All of these things reduce the burden on the social safety net, and free up resources to be used elsewhere in our society. 

ADUs are likely to face pushback from people who don’t like the thought of single-family neighborhoods allowing multiple families to live separately on the same plot. Educating these people about the benefits of affordable housing could help garner support for ADUs. 

As Central Ohio grows over the next 25 years, policymakers are going to have to come up with creative ways to make housing more affordable. New construction of affordable housing is likely going to be required at some point, but we don’t have to wait that long to improve conditions. 

ADUs are not going to solve the housing crisis by themselves. However, if policymakers change the zoning rules and allow ADUs to be built, we could see short-term improvements. Policymakers should be looking for low-cost ways to bridge the gap until a more permanent solution can be implemented.