2002 NLPES-Fall Training Conference Notes
Measuring Outcomes and Agency Effectiveness
Cathleen Akers
Arizona Office of the Auditor General
-In 1993 and 1997 Arizona reformed its budget process. Part of this
reform process included the implementation of Program Based Budgeting.
Another aspect of budget reform included the use of strategic planning
as well as the use of performance measures.
-Executive and legislative agencies play several roles in the budget
process and the use of performance measures.
-
State agencies must submit a plan each year for programs that include the
use of performance measures.
-
Office of Strategic Planning and Budgeting (OSPB) receives and compiles
the plans from agencies and then publishes them in the Master List. The
OSPB does not review these plans or the performance measures before publishing
them.
-
The Joint Legislative Budget Committee (JLBC) does some review of the master
list once it is published but it is not systematic.
-
The Office of the Auditor General (OAG) has no formal role in reviewing
performance measures.
-The OAG does include performance measures in some of its products.
-
Report findings often include performance measures.
-
Performance measures may impact the content of recommendations.
-
The Program Fact Sheet lists measures used by agency, the alignment of
those measures with mission and goals, and the reliability and validity
of the data.
-The Arizona Office of Tourism, Arizona Automobile Theft Authority, and
AHCCCS-Quality of Care are three reports that looked at agency use of performance
measures. These reports found that the measures used by the agency often
were not appropriate for what the agency was reporting. The agencies often
did not review their own performance levels nor did anyone else look at
this.
-OAG audit teams have begun looking at performance measures earlier
in the audit process so as to avoid surprising the agency with any criticism
of their performance measures later in the audit. JLBC has begun to include
some performance measures with their appropriation reports.
Gerald Hoppman
Kentucky Auditor of Public Accounts
-Kentucky is currently involved in budget woes. There is currently no
budget for Fiscal Year 2003 or 2004. The state constitution holds no provision
for what to do when this happens. The governor's interim spending plan
for FY 2003 is being tested in the courts.
- Kentucky is also facing revenue shortfalls and Medicaid deficits.
-Kentucky has started a Performance Based Budgeting Pilot Program.
-
The Auditor's office was involved in drafting the legislation that created
this program and created an oversight role for itself.
-
4 Programs were selected to participate in the program. Each program had
to submit a performance based budget in addition to its regular budget
request.
-
The Auditor is responsible for three studies in conjunction with the pilot
program: 1) Conduct a study on the flow of information from the programs
to the governor and legislature, 2) Monitor the progress of the pilot project,
and 3) Evaluate the effectiveness of the project.
-The first study found that information was not consistently reported to
the legislature or the governor. There were also no policies for developing,
reporting, and verifying performance information.
-The second study had many findings:
1) Variations existed across the pilot program agencies as to who
submitted the budgets and how they were submitted.
2) Performance information was inconsistently reported as part of the
budget process.
3) Required performance information for new or expanded programs was
not included.
4) No policies, procedures, or guidelines existed for assessing validity
and reliability of performance information.
5) Performance information was not input into software for submitting
budgets.
-The last study found:
1) Performance based budgeting should be replicated across state government.
2) Costs of using performance based budgeting varies across programs.
3) Performance information is not consistently portrayed by all the
programs.
4) Performance information was condensed by the governor's office.
5) There was a lack of legislative feedback on performance information.
Ron Sissel
New Mexico Legislative Finance Committee
-New Mexico wanted to look at the use of performance measures by state
agencies. First was a review of the work other states had done on performance
measures. They then looked at all the performance measures that agencies
collect and then decided what they wanted to emphasize. Decided to focus
on data reliability, validity, and management use.
-Data reliability measures to what extent performance is accurately
measured. Validity measures whether the data actually reflects program
impact or public benefit. Management use simply looks to see whether agency
managers are actually using the data to asses how the agency is performing.
-The audit approach for looking at performance measures included four
themes:
1) Selecting a sample of measures.
2) Start at the program level and then work up.
3) Only go down as far as necessary in order to find appropriate data.
4) Work in partnership with the agency/program.
-Lessons learned included:
1) The audit staff needs to remain flexible.
2) Sometimes will need to rely more on interviews and less on data.
3) More documentation is often needed on the agency side.
4) Agency management needs to be involved more in the use of performance
measures.
 2002
Fall Training Conference Notes
|
