Evidence-based policymaking can arm lawmakers with information about what works.
By Allison Hiltz
Friedrich Nietzsche once said, “There are no facts, only interpretations.” But decision makers around the country beg to disagree. Not only do they challenge the premise that there are no facts, but also that they are not open for debate. It is with this mind-set that officials in all levels of government have begun embracing evidence-based policymaking.
More than a buzzword, evidence-based policymaking is becoming a movement unto itself. A recently released report from the Pew-MacArthur Results First Initiative shows that all 50 states use evidence-based policymaking in some way. Many incorporate the findings from their in-depth analyses into their budgeting decisions, boosting funding for programs that not only work, but also provide a positive return on the tax money invested.
Since 2011, leaders from 24 states and eight counties have adopted evidence-based policymaking procedures with help from the Results First Initiative, a project of The Pew Charitable Trusts and the John D. and Catherine T. MacArthur Foundation.
According to project director Sara Dube, Results First works with states and localities to help them adopt an evidence-based policymaking approach so decision makers can identify and fund programs that yield high returns on investment. Under the initiative, state agencies inventory and analyze programs—from criminal justice to child welfare—to determine their effectiveness and their financial impact. The term “evidence-based policy” is fluid, and not all states, departments and agencies adhere to the same definition. Generally, the term refers to supporting policies based on careful, unbiased, scientific analyses of their effectiveness. Several states have statutes, administrative codes or executive orders defining how and when evidence-based policymaking is required.
Examples abound. Mississippi codified its definition of “evidence” and and created criteria to clarify what constitutes “evidence-based.” Oregon requires certain human-services agencies to direct at least 75 percent of their funding to evidence-based practices. California now requires results to be reported along with expenditures for its drug and alcohol programs.
The Shift to Tighten Belts
There are various reasons why states began shifting toward this method of policymaking, but one shared event had an influence on all: The Great Recession.
As revenues began to decline in 2008, state lawmakers sought ways to tighten their belts without losing their shirts. Although strategic approaches to decision making were not a novel concept, they oftentimes focused solely on the budgeting process. Performance-based budgeting, or budgeting based on results and cost-benefit analyses, or weighing a program’s financial pros and cons, were not uncommon.
Washington blazed the trail toward data-driven decision making in 1983 when it created the Washington State Institute of Public Policy and charged it with being the nonpartisan research organization within the legislative branch dedicated to improving citizens’ lives. Since then, it has evolved into a central component of Washington’s government, conducting valuable, candid cost-benefit analyses for lawmakers.
The institute, for example, just finished analyzing the benefits and costs associated with long-acting injectable medications versus those taken orally for opioid and alcohol disorders. They found that injectable naltrexone, while effective, costs nearly four times more than methadone pills, resulting in a negative return on investment. Each naltrexone treatment costs more than the benefits it yields (benefits include health insurance savings and future earnings), whereas methadone was relatively inexpensive to administer and generated $2.22 for every dollar spent.
The distinction between an effective treatment and a cost-effective treatment is a key one, says Elizabeth Drake, a senior research associate at the Institute of Public Policy. “The important thing to remember is that there are many treatment options that work, though some may be more expensive than others. Our goal at the institute is to monetize policy options, not to make value judgments, and enable lawmakers to make choices that best serve their individual communities.”
Despite differences in the ways and rates at which they’ve adopted evidence-based policymaking, many states are moving forward with plans to incorporate more of it into their budgeting practices. And all share a commitment to support programs that are proven to work and that allocate taxpayer money in the best way possible.
Minnesota Means Lean
Minnesota launched its Office of Continuous Improvement a decade ago to bolster government services and cut excess costs. The state’s Enterprise Lean Initiative, one of several programs within the office, works to maximize efficiency by training its employees to spot opportunities to streamline procedures or methods to save taxpayer money. The program has proved successful.
The Department of Human Services, for example, reduced staff time spent annually on appeals from the elderly and disabled concerning personal-care attendants, from an average of 87 days to less than a month. The reduction saved the state about $960,000 yearly.
The Minnesota Housing Finance Agency saved money by trading its stamps and envelopes for emails. By switching its delivery of payment vouchers to residents participating in a federal rental assistance program, the agency garnered more than $16,000 in annual savings.
In 2010, the governor kicked off the Better Government Award to recognize agencies that continue to improve services while cutting costs. Some of the winning accomplishments include:
- Annual savings of $400,000 from adopting an e-filing system for business tax returns.
- A 62 percent reduction in the time it takes for road construction projects to receive approval by increasing interagency collaboration.
- Annual savings of $630,000 by streamlining the mail-processing system to lower the amount of returned mail.
Until 2015, Minnesota’s application of evidence-based policies and return-on-investment analysis was applied informally on an agency-by-agency basis. It saved a lot of money, but it often had more to do with streamlining services and improving delivery, than with any serious cost-benefit analysis. In 2015, Minnesota worked with Results First and began digging into the details by inventorying programs to ensure they were both evidence-based and providing a positive return on investment.
The initiative received bipartisan support from both legislative and executive branches and is housed in the Minnesota Management and Budget office. “It offers strong, reliable data on cost-effectiveness and program quality, informing the work of policymakers and helping us to provide the most efficient and effective services to Minnesotans,” says Myron Frans, the budget office commissioner.
Today, Minnesota Management and Budget is coordinating with staff from all levels—executive, legislative, agency, counties, key stakeholders—to inventory key programs to ensure the services offered are financially efficient and maximize their potential based on the available evidence.
New York’s Hub Proves It
New York’s Division of Criminal Justice Services is a hub for criminal justice data tracking and analysis. It examines everything from crime trends to alternative-sentencing programs, then targets funding to the programs with proven records of positive results.
Recently, the division changed the way it invests in alternatives to incarceration. Using the Results First process, the division analyzed which programs were most likely to reduce recidivism and which provided states the most bang for the buck.
Division staff wanted state lawmakers to see how the process can result in evidence-based policies that have a measurable effect, rather than just produce forecasts for the next decade.
The analysis showed that several of the alternative programs were successfully turning lives around, with the potential of saving the state money down the road.
Community-based employment programs, for example, could generate a taxpayer return on investment of $2.58 for every $1 spent. The same held true for cognitive behavioral interventions, which returned $2.52 for every $1 spent.
The results impressed Marc Schabses, the division’s cost-benefit coordinator. “We want our state agencies and our Division of Budget to think of the cost-benefit analysis process as a routine part of going through the program selection and budget assembly process,” he said in a testimonial for the initiative.
Sun Shines on Florida Savings
In 2003, at the direction of the Florida Legislature, the Office of Program Policy Analysis and Government Accountability set out to determine the cost-effectiveness of a new program. It was designed to offer alternative, therapy-based programs to nonviolent juvenile offenders in the community, rather than in residential facilities. Armed with research, the office estimated the program, named Redirection, would save the state $1.7 million.
In reality, the amount was much higher, generating an estimated $2 million in cost savings its first year, reduced recidivism, and was so effective it was expanded. Within five years, Redirection generated $51.2 million in cost savings and avoided $5.2 million in costs, in part, because of the resulting lower recidivism rate.
The Department of Juvenile Justice took a series of steps in 2006 to ensure a consistent adherence to evidence, including defining the levels of evidence, developing a guide that ranks programs by effectiveness and offering ongoing support to officials as they plan and begin using the new program.
Until 2015, these efforts were largely led by state agency directors and state legislators, but in 2015 Florida Governor Rick Scott (R) issued two executive orders directing the corrections, juvenile justice, children and families, health, and health care administration agencies to collaboratively evaluate and streamline their services and budgeting.
Officials from the various agencies are working with Results First staff to evaluate programs and direct funding based on what they find. Currently, they are taking an inventory of all juvenile justice programs.
More Than Just One Way
As states increasingly shift toward evidence-based policymaking, there’s no one way to do it—particularly when it’s combined with cost-benefit analyses. For some, the shift begins at the department or agency level and migrates upward. For others, it begins as an executive order or via legislation and cascades outward, to departments and agencies.
There are times when things will not work out as expected, and times when they’ll turn out better than anticipated—when the findings will confound you, but the facts will convince you.
How state lawmakers apply evidence-based methods vary, but they all share the common goal of trying to make government more effective and efficient.
“Evidence-based policymaking can help guide government policy and funding decisions,” says Results First project director Sara Dube. “By focusing limited resources on services and programs that have been shown to produce positive results, governments can expand investment in cost-effective programs, consider reducing funding for ineffective programs, and improve the outcomes of services funded by taxpayer dollars.”
Even Nietzsche might agree with that, but probably not.
—Allison Hiltz is a policy associate for strategic initiatives at NCSL.