In the fast-paced world of state legislatures, what can policymakers do to ensure they have access to the best available information when making decisions? Representatives from three organizations say that prioritizing rigorous evidence can illuminate the complex questions states face and help identify solutions to improve lives.
In an NCSL Base Camp session moderated by David Yokum, professor at UNC-Chapel Hill’s School of Data Science and Society, panelists discussed the gap between research and the state decision-making process. Even when valuable information exists, there’s difficulty in getting it into the hands of those who can use it.
Justin Milner, executive vice president of evidence and evaluation at Arnold Ventures, says a key challenge is making research relevant to policymakers. “How do we take academic insights and apply them in the real world?” he asks, adding that answering that question is important for tackling issues ranging from the opioid crisis to housing stability.
For Patrick Carter, vice president and state policy lead for Results for America, it’s about states moving from “making educated guesses about what works to making decisions based on evidence.” To support a shift from being “well-intentioned to impactful,” Carter suggests that states define evidence at the outset and make clear how evidence information will be collected and prioritized when making budget and grant decisions.
Closing the Research and Evidence Gap
Arnold Ventures’ Evidence and Evaluation team invests in research that seeks to answer questions about what works, and funds intermediaries that connect those who generate evidence with those who can use it in the policymaking process.
J-PAL North America’ State and Local Innovation Initiative was launched to demonstrate that state and local governments could not only use and generate evidence on their own to answer policy-relevant questions, but also help build a national culture of evidence-based policymaking.
Results for America creates tools and resources, such as a state policy guide to leveraging federal uniform grants, a state evidence-based budgeting guide, and a summary of the potential impact of investing opioid settlement funds in strategies that work.
Louise Geraghty, government partnerships lead at J-PAL North America, recommends that states start by looking at existing literature and evidence. A generalizability framework like one developed by J-PAL can help policymakers to determine whether evidence generated in another state could be applied to an issue in their own state.
According to an article in the journal BJOG, states can also generate their own evidence by conducting randomized controlled trials, or RCTs—often called the gold standard for determining the effectiveness of a program or policy. “Conducting research can illuminate the work that states are doing across policy areas,” Milner says. However, RCTs can be costly and time-consuming and often require specialized expertise in design and implementation.
Fortunately, states can find a cheaper option in existing administrative data, according to an OPRE Report from the U.S. Department of Health and Human Services. By promoting data sharing and building data infrastructure, states can tap into this underutilized resource for program evaluation. “Being able to better understand how data speaks to one another across agencies helps us understand how to improve programming and serve communities,” Geraghty says.
The federal government also offers funding and support for evidence building. Under 2024 guidance from the Office of Management and Budget, states are permitted to use a portion of federal grant funds for data and evaluation purposes. (The process is explained in a policy guide from Results for America.) The opportunity creates “a new world in terms of what states and local governments can do with federal funding” to improve outcomes for residents, Carter says.
Lastly, organizations such as the three represented by the panelists provide resources and funding for rigorous evaluations. J-PAL North America connects experts with state governments to answer research questions and provides training and technical assistance. Results for America provides no-cost support for decision-makers and develops resources like its recently released state evidence-based budgeting guide. In addition, Arnold Ventures issues requests for proposals to fund RCTs and other causal studies to evaluate social programs across the policy spectrum.
Though the conversation has shifted from whether states should invest in evidence capacity to how to do it most effectively, there is still an opportunity to strengthen these efforts, Geraghty says. The panelists had several suggestions for how state legislatures can support this work. Directly funding evaluations and evaluation capacity could allow states to do more on their own. Embedding evidence and data-sharing in legislation could help state leaders sort through the noise quickly when making tough decisions. States can also lay the groundwork by defining terminology around evidence and standardizing the way it should be applied.
With so many factors at play in the legislative environment, evidence cannot be the only driver in decision-making. But systematically bringing timely, relevant evidence to the discussion can lead to more informed decisions and improved outcomes for states, the panelists say.
Carrington Skinner is a senior policy specialist in NCSL’s Center for Results-Driven Governing.