NCSL sat down with Alabama Senator Arthur Orr (R) in December 2020 to learn how data and evidence have helped guide lawmakers’ decisions and investments. As Senate appropriations chair and member of NCSL’s evidence-informed policymaking work group, Orr is a recognized leader in using data and evidence to inform policy decisions in his state.
In 2019, Orr sponsored legislation that established the Alabama Commission on the Evaluation of Services—a bipartisan and cross-branch commission charged with advising the legislature and the governor on evidence-based policymaking (EBP) and the effectiveness of state-funded services. In this interview, which has been edited for length, Orr explains how engaging the executive branch and mentoring legislators are helping to build a culture of evidence-based policymaking in Alabama.
What brought you to advocate for data-driven decision-making—and what has kept you so engaged?
When I became an appropriations chair about 10 years ago, the [American Recovery and Reinvestment Act of 2009] stimulus had run out on states and so we were pretty much left on our own. We began cutting and cutting. As a part-time legislator and appropriations committee chair, it’s difficult to understand which programs are worth it and which ones are not. As a budget chair, you sit there and ask, which one has merit? Which one doesn’t? How do we know where to put our money, and what makes sense for our state?
Of course, the agency heads believed their programs were effective and deserved continued funding. When I would ask them how they know, most of them couldn’t produce much evidence to back it up. Over time we began writing into our budgets a request for a report on the effectiveness of programs. That perhaps was not the most efficient or the most conducive way to get to the truth. They would put together something that reported a program in glowing conditions, or they would have a particular lobbying group or contractor complete a report, bring it in and drop it down on my desk and say, “See, this independent study funded by the contractor of the program says this program is well worth the money we invest in it.”
At least for myself, I grew cynical. So that brings us to our current status: How do we have an independent group that will evaluate the programs and services of our government, and be a fair arbiter of what is working well for the dollars we are spending? And are there programs that could be more effective if they had more funding? Or are there programs that quite frankly need to be put to rest? That’s the reason we started the Alabama Commission on the Evaluation of Services, or ACES. We can make much more precise, targeted funding decisions knowing the programs that are out there and having them evaluated. Having the capability to understand the programs and their impact can give us a more tailored approach.
As appropriations chair, what are the biggest challenges you face with respect to having the data you need—and how do you address them?
We still require reports from these agencies that we instituted several years ago. I’m a firm believer that you can’t improve what you don’t measure. If we’re not measuring, it’s very difficult to improve because you don’t have a baseline, you’re just throwing money at the problem and not measuring the results.
Several years ago, Alabama embarked on a SMART budgeting initiative that required government agencies to collect data, but unfortunately in many instances it was not qualitative. For example, a Department of Human Resources child services worker was required to record how many children and families they dealt with each day, but there was no data to show what happened to the child or family after the fact. The same with prisons. We would learn how many inmates were in the welding class, but nothing about whether they returned to prison after release, and whether they got a welding job after prison. My point is that our data must be qualitative, not just quantitative.
Can you tell us more about ACES and how such a structure can help policymakers make data-driven decisions? It’s early, but what are some of the commission’s early steps?
We brought our executive branch to the table early on. With ACES, we started a small agency and commission made up of the executive branch and legislators—that is more difficult at the outset but long term will give us more strength. It’s all for the betterment of our state. You need that cooperation and not confrontation with the executive branch, you need them to be on board and to buy in. We need to see this as a partnership because they’re responsible to the taxpayers just like the legislature is.
We’re early in the process. In year one, we tasked our team of four to look at suicide prevention programs. The report said we have a prevention plan, but it’s vintage early 2000s, and we really haven’t done much since. We have a hotline and a few other things, but now we’re in COVID, and suicide rates are going up. They recommended several different steps that we could take as policymakers to improve. As we focus on our suicide delivery programs through our Department of Public Health, Department of Mental Health and Department of Veterans Affairs, they know what we’re doing, and why we’re doing what we’re doing. They have the report and they have the recommendations, so they don’t feel like they’re getting blindsided or we’re out to have a “gotcha” moment. We want to work together with them and improve our suicide prevention programs here in the state and move our numbers back down.
This is a long game. Unfortunately, what we’re finding is a lack of data and how difficult it is for our ACES team to get accurate data that will help them draw the conclusions they need to draw. We evaluated suicide prevention and that took one year, and that is just one program out of scores and scores. This coming year, we’re looking at prison education and community corrections. Are our education efforts reducing recidivism? Are our inmates finding jobs after they leave? Or do they have a credential that’s useless to them in the marketplace? With pre-pandemic unemployment rates below 3%, we need every person, including inmates coming out of the correction facilities, to have some basic skillset that they learned while they were in prison.
How do you explain to the executive branch and to other legislators why using evidence matters? How do you “build the bench” and bring other policymakers who may be skeptical along?
It took some lobbying here in Alabama to get the executive branch on board. It helped to have The Pew Charitable Trusts—an independent group—come to the table, to be there with the governor, to show that evidence-based budgeting is being done in other states and is providing positive results. There’s also a role for NCSL because people trust you, even on the executive branch side. NCSL can provide the introductions or give credibility to initiatives like this. Governors shouldn’t be fearful and their staffs shouldn’t be fearful and their agency heads shouldn’t be fearful. This is good for everybody.
Those of us who are interested in evidence-based budgeting and policies need to identify other members in both chambers who are interested in furthering this initiative and make sure those members are placed in positions on our ACES commission for years to come. In Alabama, executive term limits are eight years max and they’re gone. Legislators can serve decades, and so they can have the continuity that a governor doesn’t have.
I’m already identifying legislators I can bring on board who are freshmen who show an interest in the fiscal and efficiency side of government and whom I can mentor to get them plugged in. I’m looking for someone who is dedicated to the grinding work of the gears of government and not someone who wants to embarrass the executive branch or seeks attention rooting out all this waste, fraud and inefficiencies in government. It’s someone who sees the larger picture and the longer game of slowly but surely building a better state government just by its deliverables, and measuring those deliverables, and making sure that we’re holding our government accountable and getting the productivity and efficiencies that we expect, and that taxpayers expect.
What guidance can you share with other policymakers who want to take steps to advance EBP in their states? Are there simple steps you took early on or questions you ask to assess whether there’s evidence behind a funding request?
First, I want to know who performed the study, because many times you find an agency or its performed studies and they’re always glowing. The initiative to be self-critical is not very high. Rarely does an agency head request level funding. It helps to have independence. ACES staff are government employees, they’re not a contractor or an accounting firm coming in to do forensic audits or anything. To me, that makes their findings so much more valuable.
The investment is not that much, considering the value it can bring. We’re not trying to have a big, expensive agency—$500,000 to $600,000 can get you started. It’s just about starting the process and realizing you cannot be impatient. You cannot think that immediate revelations are going to happen and you’re going to find all this waste, fraud and abuse, or productive programs that need more money, because you’re looking with such a small lens into a part of a massive state government. Those successes will happen in due time to build a more efficient government. And, over time, as you build the rapport with the agencies, I’m convinced you’ll see the recurring value in the years ahead. That’s value for the taxpayers, and that’s what we all—Republicans, Democrats and independents—want as state legislators.