Fiscal Affairs Program

Lessons Learned: What Experience Teaches about Performance-Based Budgeting and Reporting

Ronald K. Snell
August 2000

This report is based on interviews carried out with legislators and staff in Florida, Minnesota, North Carolina, Oregon and Texas in the course of 1999. NCSL staff carried out these interviews in conjunction with staff of The Urban Institute, with the support of funding from the Alfred P. Sloan Foundation.

Appendix 1 reports specific responses from legislators and staff to our question "What recommendations would you make to other states?"

I. Requirements

Legislators and staff in each state agreed that results-based government requires a permanent institutional commitment from the governor, legislature, and agency administrators, a statutory base, adequate staff resources, existence of an oversight agency, investment in data management, and recognition that fundamental change requires time.

1. Strong executive leadership is essential.

Texas's success in implementing performance budgeting has in part resulted from a commitment to statewide and agency strategic planning on the part of the Democratic administration of former Governor Ann Richards and the current Republican administration of Governor George W. Bush.

In other states, respondents identified the absence of such a commitment as a weakness. For example in Minnesota, legislative staff identified the executive Budget Office's wavering commitment to issuing appropriate budget instructions as a weakness in the performance reporting process. Staff of the Minnesota Auditor's Office commented that that the failure of past governors to do more than generally encourage performance evaluation had meant a lack of specific instructions on what agencies should do.

2. Legislative commitment is essential.

Respondents in each state emphasized the need for a joint, coordinated commitment from the legislative and executive branches. The legislature has its own role to play in the expression of a commitment. Two particular points that stood out are:

A legislative role in selecting performance indicators is important to assuring that indicators of performance are relevant to legislators' concerns. Legislative involvement in the process can help address some legislators' concerns that performance reporting is only a public-relations process for agencies. Exemplary practices are those in Florida, where the substantive (or policy) committees review the agencies' recommended indicators and enact them. The Florida legislature attaches a few indicators directly to language in the appropriations bill, and in 1999 enacted longer lists of indicators in separate legislation (the "Implementing Bill"). Oregon legislators spoke enthusiastically about performance reporting only in connection with the transportation and education programs in which the legislature had established performance indicators.

Bringing substantive committees into the process strengthens the process.. In legislatures where a large proportion of the total membership is involved in budget committees, it may not be necessary to seek ways to bring other legislators into the process. However the Florida model, in which the substantive committees review and agree on performance indicators for the use of the budget committees, has the advantage of giving every legislator some involvement in the choice of performance indicators. Florida staff commented that, in addition, this process has produced a performance management system, not just a performance budgeting system.

Support from one branch of government is not enough. The executive and legislature bring different strengths to the process. The executive has to provide central direction and enforce agency commitment. The legislature has to demand and use performance information in order to communicate to agency personnel that performance reporting is not simply make-work. Minnesota and North Carolina, where there has been executive but not legislative commitment, make this clear. And where there is legislative commitment in the face of executive indifference (as in New Mexico) it is very difficult for the legislature to maintain continuity of policy and to build support in state agencies.

3. More staff may be needed.

The two states that come closest to performance budgeting and that have gone the furthest in performance evaluation, Florida and Texas, have done so in part because they have staff agencies with the resources to help legislators and state agencies develop, validate, and use performance indicators.

It would be an oversimplification to say that success in governing-by-results is proportionate to the size of legislative staff. But it is clear from staff comments in interviews that the relatively small numbers of fiscal and other committee staff in Minnesota and Oregon have hampered efforts to build an effective performance measurement and performance budgeting system in those states. In Minnesota, for example, the Legislative Auditor's Office has produced some exceptionally useful reviews of agency performance measures, including practical recommendations on improving the design of measures, but has lacked the staff to sustain this essential effort. The legislative fiscal staffs in those states do not have enough staff to analyze performance results along with regular budget analysis.

In Florida these responsibilities are divided among staff for the fiscal committees, substantive committees and the legislative Office of Program Policy Analysis and Government Accountability (OPPAGA).

The Florida legislature has divided the committee staff workload by assigning the review of agency performance measures to substantive committees, unlike other legislatures, so that the fiscal staff do not have to carry the full load. In addition to the budget committees in each chamber, the Florida Senate has a Fiscal Policy Committee and the House has a Government Operations Committee that are involved in the performance reporting and budgeting processes (the House Government Operations Committee chair, for example, in 1998 developed Florida's unit-cost measurement legislation). Each committee is staffed with a committee staff director and assistants.

OPPAGA, with more than 95 staff, has resources to carry out these responsibilities:

Performance audits and policy reviews of state government programs.
Follow-up reviews that determine whether agencies resolved problems identified in earlier reports.
Technical reviews of agency performance-based budget proposals.
Performance evaluation and justification reviews of agencies operating under performance-based budgets.
The Florida Government Accountability Report, an Internet electronic encyclopedia containing descriptive and evaluative information on state programs.
Technical assistance to legislative committees.

In Texas, such responsibilities are centralized in two agencies, the Legislative Budget Board (LBB) and the Auditor's Office (a legislative agency in Texas).

The LBB, which has over 110 staff, is primarily responsible for preparation of the legislative budget, appropriations bill, and fiscal notes, but in connection with the state performance budgeting process it analyzes performance reports, assists agencies in the preparation of strategic plans, and tracks performance developments among state agencies when the legislature is not in session. The LBB staff also maintain the Automated Budgeting and Evaluation System for Texas (ABEST), which allows agencies to report performance information electronically.

The Auditor's Office examines the accuracy of performance measures and how management uses performance information to manage operations. The office also assesses the accuracy of reported performance measures; verifies internal controls in order to provide safeguards over the collection and analysis of performance measure. Additionally, the office examines how agency management uses measures to adjust operations to better achieve expected results.

4. An oversight agency is needed.

This point has been made by legislators and staff in a number of states. This role is largely met by the Florida and Texas agencies described above and by the role of the executive budget office in Florida. However, legislative staff in Florida and other states (excluding Texas) have expressed concern that the executive budget offices have not exerted the needed leadership for agencies. Texas's state agencies are accustomed to a legislative budgeting system that provides leadership that in most other states is the responsibility of the executive branch.

The lack of such leadership is apparent in Oregon and Minnesota. Oregon's Progress Board does not have the authority to exercise leadership and legislators view its reports as remote from their responsibilities. The governor has not taken on this role, and this helps explain why Oregon's benchmarks don't have much impact on the policy-making process. Minnesota staff reported that the executive Department of Finance, which has been responsible for producing agency instructions on the design of performance reports and their integration with budget requests, has suffered from variable and inadequate funding for this responsibility.

On the other hand, the activity of the North Carolina Office of State Budget and Management has shown how decisive executive leadership can foster performance management and reporting even in the face of legislative indifference.

5. It may be necessary to invest in more comprehensive information systems.

This point has emerged in the interviews in each of the five states this project has focused on as well as other states involved in the process. The concern is to facilitate the collection, analysis and presentation of performance data and to link the data with budget information. Texas has developed a unified budgeting and performance reporting system (the Automated Budgeting and Evaluation System for Texas or "ABEST"). Oregon is developing a comparable budgeting and reporting system. This is an important step not only to improving the quality of information available to policy makers, but to institutionalizing government by results. Florida respondents indicated that the lack of an appropriate budgeting and information retrieval system has frustrated efforts to build a true performance-based budget.

6. It all takes a lot of time.

Florida's OPPAGA has documented the amount of time required for an agency to implement performance-based program budgeting in its recent review of the Florida process, PB² Status Report, Fiscal Year 1998-99. The timeline shows that it takes about 18 months for an agency to design and win approval of its proposed program structure and performance measures from the governor and legislature and additional time for the program and measures to be incorporated in legislation so the agency can operate in accord with them. About four years pass from the beginning of the process in agency workshops to its completion with OPPAGA's review of the agency's first year under performance budgeting.

The time-consuming nature of the process is one of the reasons that institutionalization of the process is so important. Legislative turnover has increased in recent years, as party control of legislatures and governors' offices has changed, as term limits have begun to take effect in a number of state legislatures, and as long-term legislators retire. The implementation of governing-for-results can outlast the terms of the policy makers who initiate the program.

II. Performance Indicators

Legislators and staff raised numerous issues about the choice of indicators and reports of performance. The most frequently heard complaints were that indicators were not well-chosen to reflect legislators' concerns, that too much information was reported, that presentation was not well-planned and that the reported information was unreliable. Some state governments have adopted practices that address these issues.

7. Legislators have to be involved in selecting performance indicators.

One reason is to make sure the indicators are relevant to legislators' concerns. Agency personnel and legislators can have different ideas about what is important about an agency's work. Another reason is to address a concern we frequently heard, that agencies pick indicators to show themselves off at their best. It's clear that indicators have been more useful to legislators in Florida and Texas, where legislators review them, than in Minnesota and Oregon, where they don't have that opportunity.

8. Indicators should be linked to appropriations.

This recommendation was made by legislators and staff in a number of states. Of the states in which interviews were held, Texas is the only state where output and outcome indicators are specifically linked to funding in the appropriations bill. Florida has in the past included more indicators in the appropriations language than it did in 1999, when most of them were enacted in a separate bill. Legislators and staff suggest that agencies are most likely to heed indicators when the linkage to funding is explicit. The Florida legislature's OPPAGA has recommended a format for clarifying the linkage.

9. Agencies should specify how any additional funds they request would affect their output and outcome measures.

Legislative budget review tends to focus on changes in funding rather than on the base (the amount appropriated for the previous budget cycle). Specifying the changes in agency performance that could be expected to result from a change in funding would provide legislators with relevant and pertinent information of immediate use in policy making decisions.

10. Use of unit-cost measures strengthens the process.

The Texas appropriations bill already includes some unit-cost measures, and Florida legislation has required them to be developed for all agencies and programs. The program budget Governor Jeb Bush submitted to the Florida legislature in January 2000 built unit cost measures into the proposal for the first time. OPPAGA has recommended that unit-cost measures be reported along with other measures, saying that

Unit costs identify the resources needed to produce outputs such as providing a single unit of service or providing a set of services to a single individual. The legislature could use unit cost information to assess the relative efficiency of program operations or to determine the relationship between changes in the cost of program services and the outcomes obtained from these services.

11. The number of indicators should be limited.

Legislators make use of indicators, particularly in Florida and Texas, in policy discussions, but report that they are sometimes hampered by the volume of indicators, the amount of material reported, and uncertainty about the quality of the information reported.

Legislators and staff can feel overwhelmed by the number of indicators associated with agency activities and the volume of material reported. Minnesota legislators have repealed the requirement for separate agency performance reports in part because of the bulkiness of the reports. The legislative auditors' staff in Minnesota reported that performance reports were too long to be assessed biennially. Texas legislators who were asked whether there are appropriate numbers of performance measures and targets in the appropriations act and the performance-based budgeting system divided about 50-50 on the issue. Their comments indicated a need either for fewer, better defined, or better prioritized indicators.

The number of indicators is large because of the size of state government, the variety of its activities, the different interests of different legislators and, in Minnesota at least, a fear among mid-level agency officials that a program without indicators is a program that legislators will disregard--a variant on the Osburne-Gaebler dictum that "What gets measured gets done."

12. Adequate staffing is probably the only resolution to the problem raised by large numbers of indicators.

Oversight agencies and state budget offices deal with this issue by prioritizing indicators and using a relatively small number of them in budget recommendations and legislation. Legislative staff responsibilities in Florida, Texas and Louisiana include tracking agency performance and reporting strong or weak performance in relation to specific expectations to legislators.

III. How Legislators Use Performance Information

13. Legislators find performance information valuable in the policy making process.

Legislators we interviewed spoke favorably of the potential or actual value of performance reports in the policy making process. Some of those who found performance reports lacking in value had hoped that the reports would facilitate budget cuts and smaller government, and were disappointed that performance reporting does not readily lend itself to that use.

Texas surveyed legislators about performance budgeting in October 1998. Twelve percent of respondents reported that they use performance reports as a tool to accomplish specific tasks. In the words of one respondent this meant using indicators and reports as a way to ensure that an agency would perform specified tasks. Fifty-nine percent reported that the information is most useful as a general guide to overall agency performance, which meant in the words of one legislator, "monitoring the overall effectiveness of an agency" and for another as a way to confirm that legislative intentions are followed and that programs are effective. Twenty-nine percent reported that they use the information both ways.

It is clear from the same survey that legislators view performance indicators and reports as only one of a number of sources of information about agencies. One legislator remarked, "Performance-based budgeting does not stand alone as a specific budgeting tool. It is only one of many analytical activities and is most useful as a guide."

Florida legislators said much the same thing--that Florida's performance-based budgeting system has been useful in the legislative process although it has had less direct impact on the budgeting process than expected. One committee chair noted "It helps in the process. It has improved the working relationships of the substantive and appropriations committees." Another legislator commented that it is useful in helping set priorities. A senior House staff officer commented that legislative involvement is growing because substantive committees see establishment of measures as a way to affect the appropriations process, as changes in performance measures can require changes in appropriations. But like Texas legislators, they feel, as one said, "It is a piece of the process--not a panacea."

14. Incentives and disincentives have not worked well in practice.

Legislators and staff generally support the use of incentives and disincentives to improve agency performance, but developing an effective way to do so has been difficult. The laws in Florida and Texas (and in a few other states) allow the use of incentives and disincentives, but actual use has been very limited.

Adjusting the amount of agency funding as an incentive or disincentive is not a very workable tool. Agencies that perform well can be rewarded with additional funding or additional flexibility in the use of its funds, but such rewards have to be weighed against the needs of other agencies, and funding is not necessarily available to serve as a reward. Reducing funding for programs that do not meet performance goals may not be an effective course; agency failures could be the result of inadequate funding and reduced funding could make the matter worse.

Incentive awards for individual employees have been suggested in some states, but state personnel laws and union agreements (where they exist) have made this a less useful tool in the public sector than in the private sector.

Louisiana imposes stricter reporting standards on agencies that fail significantly in meeting performance goals. It appears in practice that requiring reports and the use of praise and admonition may be the most effective incentives and disincentives that policy makers can use.

Appendix 1. Recommendations for State Action

The following express specific recommendations made by legislators and legislative staff when asked what they would recommend for other states to consider.

Leadership and Design:

There's a need for a permanent, joint legislative oversight body. Leadership needs to be involved for a long time - five years or so - because ideas have to change fundamentally. Commitment from powerful leaders is needed to make agencies comply with the system.--Florida, Oregon , Texas
You need commitment and policy decisions before embarking on the technical side - Florida did it the wrong way around.--Florida
Everything needs to be consistent - statutes, rules and regulations, mission statements and performance expectations.--Florida
Statutory provisions and other means to institutionalize the system may also contribute to stability.--Texas
It has to be much more than a budget process. It is necessary to have the substantive committees actively involved.--Florida
The evaluation of agency effectiveness should not be tied to the budget--concerns over agency efficiency and effectiveness should be separated.--Minnesota
Expectations should be flexible and be the result of a combined legislative-executive effort. --Minnesota

Implementation:

It takes time and leadership to get the process to work--it's easy to let it fade away. It takes time to make the measures meaningful. Measures need to be tied tightly to the budget to catch agency attention. The process is incremental; patience is important.--Florida , Oregon
Incentives and sanctions are essential, but they can be subtle. Without them, agencies have no inducement to comply with standards.--Florida
Provide feedback on performance to stakeholders in a timely manner. This allows for changes to be made and for stakeholders to discover cause and effect relationships between actions taken and the performance levels achieved.--Oregon .
The legislature must be organized and have sufficient staff resources.--Minnesota
Put performance measures in the appropriations bill.--Texas
Find innovative and practical ways to summarize, analyze, and format the data to make it more accessible and useful for decision-makers.--North Carolina, Texas

Data and Verification:

You have to spend money on information and data systems.--Florida
There has to be a system of verification of the reported data, technical assistance on the construction of performance measures, and evaluation of the validity of measures.--Florida
Make sure to note the limitations of any measurement data--Oregon
Keep in mind that not all activities can be measured.--Oregon
Link activities being measured to some desired outcome.--Oregon