Issue Brief: Accurately Evaluating State Child Support Program Performance

Child Support Project

Issue Brief: Accurately Evaluating State Child Support Program Performance

By Teresa A. Myers

While declining welfare caseloads and expensive federal mandates are prompting even closer examination of state child support performance, legislators, staff and administrators are struggling with how to best gauge the performance of a state child support program. Texas has recently completed a comprehensive audit of its child support program and other states, including Colorado, New Mexico and Tennessee are conducting evaluations currently. In previous years, a state's collection rate was the barometer for success, and while that statistic is still noted and necessary, experts are recognizing that an accurate evaluation of performance must have far more depth and breadth than a single number provides. States are experimenting with performance evaluation techniques that offer a much more complete examination of the state program and clearly highlight successes and shortcomings in the programs.

Federal Performance Criteria

One of the most obvious gauges of any child support program is its ability to deliver support to the children relying on it. Until recently, federal child support incentive funds were narrowly based on a state's collection rate and its program's cost efficiency. In response to state demands for a more comprehensive review process, during the summer of 1998, Congress enacted legislation to expand the performance criteria for federal incentive payments. Beginning in FY 2000, states will be evaluated for federal incentive funds based on a state's performance in five areas:

The paternity establishment percentage;
the number of cases with orders;
the number of cases receiving payments;
the number of cases receiving payments on arrears; and
the ratio of collections to costs.

State officials nationwide agree that the new performance criteria will present a more accurate record of state child support performance. In fact, many child support directors have been using criteria like these for internal evaluations for many years. They point out that these performance criteria permit greater depth of evaluation, while still retaining the simplicity necessary for easy evaluation and analysis.

Some state policymakers and advocates want to look at an even broader set of factors when evaluating their state child support program. For instance, the federal criteria do not account for payment processing performance, i.e. accurately accounting for money, paying families in a timely fashion, setting up appeal dispute resolutions, etc, or for ordering and processing medical child support orders. Many legislators and administrators argue that tasks such as these also should be considered.

Using Performance Criteria to Set Expectations and Priorities

Some states are using performance criteria not only to gauge performance retrospectively, but also to set performance expectations prospectively. These states have taken a business-like approach to their caseload, using a categorical matrix to establish case priorities and set performance expectations for case categories. While it may sound complicated, the matrix system is actually very simple, and borrows its premise from traditional methods of tax and debt collection. Basically, the matrix allows the state to measure performance based on a variety of case characteristics and case levels. First, a state divides its cases into categories based on characteristics of the cases (e.g. welfare vs. non-welfare; interstate vs. intrastate) or where in the collection process the case is (e.g. paternity needs to be established vs. an income withholding order that is already in place). Second, the categories are assigned performance expectations based on the anticipated likelihood of collection, which is determined based on a study of previous state collection rates for different case types. For example, higher expectations would be assigned to non-welfare cases within the state where an active income withholding order is in place.

Minnesota, using a variation on this model, identifies several reasons certain classes of obligors don't pay and categorizes cases based on those reasons, which affect the likelihood of ultimate collection. An independent state auditor in Ohio has recommended adoption of such a model for that state also. In this way, a state child support program can more realistically gauge performance and aim improvements at flagging areas.

Oregon's Performance-based Matrix

Oregon has been using a performance-based matrix since the early 1980's, but its automated matrix goes a step further. In addition to categorizing cases, the matrix also prioritizes the caseload based on collectibility. The matrix contains seven "function" categories performed on a child support case, e.g. paternity establishment, order establishment, etc. Each case moves through the function categories as it progresses toward collection. Cases within each function category are assigned a priority based on the anticipated collectibility of the case. For instance, a case in the paternity establishment category with a priority level of 1 indicates that the case file contains all the information necessary to proceed with paternity establishment, whereas a priority level of 10 indicates that the case file contains no usable information about the putative father and the case is considered essentially unworkable at this point. The priorities are established within the parameters set by federal law for case processing, e.g. cases must be opened within 20 days of being received. Each morning, Oregon child support caseworkers view a computer "tickler file" that not only tells them what function category each of their cases is in, but also which cases to work first, based on the priority number. State officials point out that the matrix is completely objective, removing any opportunity for caseworkers to not work an difficult case. Oregon officials report that the matrix system has been an invaluable tool in improving their child support service and collections.

Florida's Multi-faceted Performance Review

Officials in Florida gauge performance of their program by integrating several diverse measurement tools. The child support division was brought under a performance-based budgeting approach in July 1998. Using this approach, the state legislature requires the agency to align activities into clearly defined programs and to maintain records of performance in those categories. The legislature sets performance expectations for each category and then links child support funding to whether the agency has met the expectations. By identifying expected outcomes and outputs, the governor, legislature, and public are provided the information necessary to evaluate state agency performance and determine if tax dollars are used efficiently and effectively. Six measures were identified for FY 1998-99:

1. Percent of children with a court order for child support;

2. Number of children with a newly-established court order;

3. Percent of children with paternity established;

4. Percent of child support collected that was due during the fiscal year;

5. Percent of cases with child support due in a month that received a payment during
the month; and

6. Total child support dollars collected per $1.00 of total expenditure.

Annual performance expectations are developed for each regional office and are linked to the state agency's strategic plan and mission. The performance expectations are monitored and regions receive a monthly report on their progress. In addition, the state agency submits a monthly report to the legislature reflecting current progress toward state goals.

The child support program has also implemented the self-assessment process recently prescribed by the federal government to determine compliance and performance by the regional offices. Data is gathered on the percentage of cases that meet certain case activity requirements. This data is compiled quarterly for each region and annually for the state. If a certain requirement is below standard, corrective action plans are prepared and monitored.

Other States' Performance Evaluation Tools

Several states, including Colorado, Florida and New York, require county offices to submit monthly performance reports to the state child support director's office. In Colorado, the state agency uses the information from these reports to encourage better performance by county offices - each month it publishes a newsletter with a section comparing the performance of counties with similarly-sized caseloads. These comparisons result in friendly competition between counties, while also providing the agency and legislature with a monthly snapshot of state performance. New York's child support division relies on its counties' monthly reports to identify successful and problematic areas. The state agency staff includes "county representatives" responsible for tracking county performance. These county representatives make site visits, conduct educational and training sessions, and help local officials identify and attack problem areas early on. Officials in some states with county-administered programs are hesitant to compare counties against one another. Michigan, for instance, mainly judges county performance against each county's prior performance in a particular category area.
Wyoming uses teams to conduct "Staff Assistance Visits" to local offices. During these site visits, team members evaluate all aspects of the functions and performance of the office, including compliance with federal requirements, completion of "human relations climate questionnaires," and inventory of state equipment and property. The teams also assist the local officials in trouble-shooting problem areas and identifying additional state resources for assistance, guidance, or collaboration.
Texas monitors its child support performance through two groups: a program monitoring group reports directly to the child support director, while an auditing group within the Attorney General's office reports to the Attorney General, who oversees the child support director. Performance reports are reviewed and used in identifying problem areas and setting future performance expectations.

Experts' Suggestions for Conducting Performance Evaluations

State legislators and agency officials familiar with evaluating state child support programs typically agree on several basic suggestions for effectively gauging a program's performance:

Be very clear in setting program goals.

Child support programs were originally marketed to many state legislators as revenue-generating programs and state programs were frequently evaluated on that basis. As the welfare caseload has declined, most child support programs have had to change child support revenue expectations as more clients become non-welfare cases. Also, state legislators have grown increasingly concerned about other areas such as customer service, reliability of collection services, and expedited processing of cases. State experts caution against setting performance expectations that are not truly reflective of the goals of the program. For instance, generating state revenue and providing responsive customer service are two different goals with potentially two different outcomes. Investment aimed at enforcement activities will more directly result in recoupment of state funds from TANF cases, whereas customer service initiatives are expensive and not directly related to generating revenue. Instead, they contribute to higher levels of constituent satisfaction and may result in better collection rates in the foreseeable future. Similarly, if the child support division has been transferred from a customer service-oriented agency, such as human services, to a revenue-generating agency, such tax or revenue, then performance in customer service performance expectations should be amended to reflect that change. Finally, goals should keep pace with the program and with socioeconomic changes in the state. To this end, policymakers suggest frequently revisiting the broad goals of the program to insure that they remain current with the reality of the child support environment.

Evaluate the relationship between state investment and program performance.

In past years, lawmakers, frustrated with the perceived poor performance of their state child support programs, have been reluctant to inject large sums of additional investment into those programs. Most experts, however, expect that the more aggressive use of enforcement tools and development of high-speed automated child support systems will improve state child support program statistics. At the same time, a new report by the Center for Law and Social Policy illustrates the relationship between state investment and program performance. Based on an analysis of state data reported to the federal government, the author argues that state child support performance is directly related to funding of the program. For example, the study shows that nearly all of the best-performing state child support programs also have the most generous funding and staffing levels. Several state agencies have conducted similar studies within their states to demonstrate the potential for vast improvements in program performance if additional funding is contributed by the federal and state partners. Finally, child support is an open-ended federal entitlement - for nearly every dollar the state invests in the child support program, the federal government matches that investment with two dollars. Therefore, states can dramatically leverage their investment through this federal partnership. The relationship between program funding and program performance is likely to garner increasing attention as state lawmakers struggle to promote family self-sufficiency in an age of limited resources.

Understand the potential for cost-avoidance savings from state child support programs.

Recent studies have indicated that state child support programs are far more cost-effective, efficient, and beneficial to family self-sufficiency than previously acknowledged. Analyses conducted by Vermont's child support agency suggest that child support programs nationwide provide benefits to families at notably lower costs than either Medicaid or TANF. Several other states are currently examining the potential savings to other state programs that may be generated through a strong state child support program. A 1998 study in Washington attributed substantial welfare cost savings to good child support collections. According to the study, the state realized $6.5 million in welfare savings during two study periods. Washington also found that although child support was not significantly related to people exiting the state's welfare rolls, it did substantially increase how long they stayed off welfare. Last year, Michigan's child support division measured the effect of medical support orders, now a mandatory part of child support orders, on Medicaid. State officials report that these medical support orders generated state and federal savings of $1,550 per person in FY 1997 - a total savings of $228 million in one fiscal year. These early findings suggest that state investments in child support programs may have benefits to the state beyond the checks received by custodial parents.

Be careful not to compare dissimilar entities.

State child support administrators and legislators with considerable knowledge of child support systems nationwide argue that states should not compare their program's performance to a state with a program operating in a completely different political landscape or cultural/socioeconomic context. For instance, the state of Texas has a large caseload, shares an international border, and has vast cultural and socioeconomic diversity among its residents - comparisons with a small midwestern state or wealthy northeastern state might prove counter-productive. The same can be said for comparisons between local jurisdictions. Legislators in several states with county-administered child support programs caution against comparing large counties to small counties in terms of performance.

Link financial incentives to performance and don't be afraid to penalize poor performers when necessary.

State legislators and some state administrators in states as diverse as Colorado, Florida, Michigan, Montana, Ohio, and Virginia report that the most effective means of enhancing performance is to link it to receipt of funds - and follow through with close monitoring and sanctions for poor performance when necessary. Officials stress that performance-based funding without monitoring or a real possibility of financial penalties is meaningless.

Recognize that every statistic is a snapshot of the state caseload at a particular moment in time.

State officials also emphasize the importance of understanding the reality behind any given set of statistics. For instance, a 100% collection rate for a state's active caseload is impossible to achieve because any given caseload will contain cases that have just been activated and are still awaiting a court order or enforcement services. Or a paternity establishment rate may be higher or lower for a given period of years due to changes in the nonmarital birth rate in the state.

This suggestion is supported by a recent study by the Urban Institute, which examined rates of child support receipt among single mothers. One finding of the study was that although the overall rate of child support collections did not increase appreciably between 1975 and 1995 (it hovered around 20%), this was due not to poor state program performance, but primarily to the changing demographics of the parents receiving child support. For example, the number of single mothers receiving child support who were divorced or separated dropped from 82% in 1975 to 60% in 1995. During the same period, the percentage of single mothers who were never married rose from 18% to 40%. Child support is typically more difficult to collect for never-married mothers; therefore, the fact that state collection rates were stable actually indicates that states were able to retain their previous performance level. This is especially important given the dramatic changes in state caseloads, including a notable decline in TANF cases that generate recoupment of state funds. The authors credited several state child support establishment and enforcement programs, such as in-hospital paternity acknowledgment processes and tax intercept programs, with responding to the needs of changing caseload profiles. While nearly all experts agree that child support programs must be improved, these findings demonstrate how misleading a single statistic on its own can be.

Understand the impact of case closures on performance statistics.

Case closure statistics can have particularly dramatic effects on caseload statistics. Generally, once a case is closed, it is no longer included in any performance evaluation analysis. While all state programs are required to adhere to federal case closure regulations, some administrators and legislators criticize the criteria for being too stringent and counter-productive to effective case management. The federal Office of Child Support Enforcement has recently responded to some of those criticisms in drafting new rules for case closures that permit states to close certain cases that are considered unworkable. These new amendments could have serious fiscal and statistical implications for state programs. Unfortunately, recognition that case closures can positively affect performance measures also creates incentives for employing a liberal reading of the federal criteria. Experts have acknowledged that state child support directors must be diligent in ensuring that case closures are legitimate, because there are no internal checks built into the federal case closure procedures. For example, federal law permits states to close certain "unenforceable" cases, as defined in the regulations, but the federal government does not actively review closed cases for validity.

Some states are actively pursuing case closures, while attempting to insure that needed services are not terminated. For instance, New York's automated system permits the child support division to close cases, in compliance with federal guidelines, on a regular basis. For "active" cases with no recent activity, the state's automated child support system sends notices to custodial parents advising them of their case status and inquiring whether they would still like the state to pursue the case. The custodial parent has 60 days to respond. If no response is received, the case is subsequently closed.

Engage in long-term planning.

Many states are now utilizing long-term planning techniques to set goals and evaluate performance over longer periods. The complexity of child support reform measures suggests that this approach may present a more accurate portrayal of state progress than short-term benchmarks. Colorado has relied on strategic five-year plans for many years, with benchmarks based on calendar years, and annual goals proportioned to counties based on caseload size. Wyoming uses similar long-term planning tools, and both states involve the county officials in the planning and goal-setting process.

Recognize the effects that federal actions have on state child support programs.

State officials have expressed frustration at the frequent changes in federal child support laws and regulations in recent years. Both legislators and administrators complain that it is difficult to achieve state goals when new federal goals are superimposed upon states and require priority status due to the importance of federal funding. Indeed, some states have documented how new federal requirements have derailed previously successful state programs due to the need to reallocate resources to meet the federal mandates. For example, a recent independent audit in Texas revealed that the state's highly successful license suspension program has been largely stalled due to the fact that the new, federally-required statewide automated system cannot process the suspensions. State legislators were very frustrated to discover that resources had to be diverted to building a certified automation system, at the expense of one of their most popular and effective state programs.

Conclusion

Some advocates are quick to point out that child support success stories are too few and far between right now, and many state legislators and administrators may agree with them. There are more than 4 million unmarried couples in America today. Nearly 25% of children now live with a single parent and about 50% of all children are likely to spend some time in a single-parent home at some point. Millions of people are leaving the welfare rolls. Every statistic confirms that more and more families will be needing and relying on a child support check in the years to come. Our nation's ability to meet those needs will be one of the great challenges of the new welfare reform structure. While there's plenty of work to do, state legislators and agency administrators are using ingenuity and determination to create successful programs to deliver support to America's children.

Selected References:

Formoso, Carl. The Effect of Child Support and Self-Sufficiency Programs on Reducing Direct Support Public Costs, Washington State Division of Child Support, December 1998, as submitted to The Lewin Group.

Elaine Sorenson and Ariel Halpern. Child Support Enforcement: Has it Resulted in Greater Rates of Child Support Receipt Among Single Mothers?, The Urban Institute, April 1998 draft.

Sheri Steisel and Kirsten Rasmussen, The Child Support Performance and Incentive Act of 1998, National Conference of State Legislatures, www.ncsl.org/statefed/welfare/cs.htm.

Turetsky, Vicki. You Get What You Pay For: How Federal and State Investment Decisions Affect Child Support Performance, Center for Law and Social Policy, December 1998.

For additional information on state child support enforcement, please contact the Child Support Project at 303/364-7700.