NLPES Training Resources

NLPES Executive Committee Meeting

Presiding: Gary VanLandingham, Deputy Director, Office of Program Policy Analysis and Government Accountability, Florida

NLPES Reception

Presiding: Gary VanLandingham, Deputy Director, Office of Program Policy Analysis and Government Accountability, Florida

"Using Performance Information in the Budget/Oversight Process"

Description:   Managing for results is no longer just a fad. There is growing commitment to this reform effort in state government. This session will provide information on the status of performance budgeting in several states. Participants will discuss how agency performance information is used in making funding decisions and consider the issue of whether agencies should be rewarded or punished based on their success or lack of success in achieving performance goals.

Moderator:

Jane Thesing, Senior Audit Manager, Legislative Audit Council, South Carolina

Panelists:

Representative Pete Gallego, Texas Legislature, Texas

Mary Noble, Deputy State Auditor, Bureau of State Audits, California

Gary VanLandingham, Deputy Director, Office of Program Policy Analysis and Government Accountability, Florida

Notes:   The connection of budgets to performance seems obvious. However, in practice it is extremely difficult to tie budgets to performance in a meaningful way. This session explored how three states (California, Florida and, Texas) have attempted to make this connection.

Florida. Gary VanLandingham described performance budgeting in Florida which, beginning in 1994, is being phased in over a seven-year period. Goals of the Florida effort include making agencies accountable for meeting specific performance targets; consideration of performance information during the budget process and consideration of the use of rewards and sanctions based on agency performance. In FY 1999-2000, 78 programs in 26 agencies are operating under a performance budget. Gary VanLandingham described performance budgeting in Florida which, beginning in 1994, is being phased in over a seven-year period. Goals of the Florida effort include making agencies accountable for meeting specific performance targets; consideration of performance information during the budget process and consideration of the use of rewards and sanctions based on agency performance. In FY 1999-2000, 78 programs in 26 agencies are operating under a performance budget.

There are four steps in the Florida process: (1) agencies propose programs and measures; (2) the Governor's Office includes this information in the budget recommendation; (3) the legislature incorporates information into the budget; and, (4) after one year OPPAGA conducts a study to evaluate performance.

There are several significant issues being wrestled with in the implementation of performance budgeting in Florida. These include determining the appropriate number of measures to report to the Legislature. Some agencies have proposed up to 200 measures for their programs, which overloads the process because legislators and staff don't have the time to tease the meaning out of this level of detail. However, other agencies have proposed only a few measures to assess all of their activities; this doesn't give the legislature enough information to evaluate their performance. OPPAGA has suggested a three-tiered approach to resolve this issue. First, measures of broad program results should be included in the budget. Secondly, key output and outcome measures should be reported to the legislature outside the budget. Finally, detailed measures that managers can use to assess low-level activities should be maintained by the agencies.

Another issue is that the 1999 Florida Legislature directed agencies to begin reporting unit cost information. This type of data is critical and could be used to assess return on investment or make/buy decisions. However, Florida's current accounting systems do not support the generation of consistent unit cost information. Florida is considering redesigning its state accounting system, which should help resolve this problem.

While Florida law authorizes the legislature to impose a variety of rewards and sanctions based on reported agency performance results, the use of rewards and sanctions is being approached cautiously. To date, there has been a limited use of rewards and no sanctions have been imposed. While substantive committees have considered performance information, there has been limited use of performance data in budget decisions to date. However, this usage will likely increase over time as agency accountability systems continue to develop and legislators gain more confidence in the performance data agencies are reporting.

Texas. Representative Pete Gallego of Texas discussed performance based budgeting from his perspective as a member of the House Appropriations Committee. Texas is on its fourth cycle of performance based budgeting. Before performance based budgeting, the legislators thought they would receive information on past performance during budget hearings. However, agencies only wanted to talk about their future plans. With performance measurement information included in the budget documents it provides a vehicle for discussions of how agencies met legislative expectations during the previous budget cycle. Representative Pete Gallego of Texas discussed performance based budgeting from his perspective as a member of the House Appropriations Committee. Texas is on its fourth cycle of performance based budgeting. Before performance based budgeting, the legislators thought they would receive information on past performance during budget hearings. However, agencies only wanted to talk about their future plans. With performance measurement information included in the budget documents it provides a vehicle for discussions of how agencies met legislative expectations during the previous budget cycle.

In Texas, performance based budgeting is intended to be an early warning system, not a tool for punishment. The legislature has focused on incentives such as improving FTE caps or providing more flexibility for agencies that perform well.

During the interim, agencies and legislators were surveyed about the performance based budgeting system. Agencies said that they did not think that a system of rewards and penalties should be used because many factors that would impact their ability to hit targets are out of their control. According to Representative Gallego, many legislators would say the same. Recognition should be given to the need for flexibility. To guard against having too many measures consider having a rule that for every new measure that goes in an old measure must come out.

In summary, Representative Gallego said that he felt that performance budgeting had helped agencies understand the relevance and importance of the legislature in the process and that it had been helpful in Texas.

California. Mary Noble said that California does not use performance budgeting to any great extent. The State Auditor is required by law to review certain programs and sometimes request are made by the budget committees for program reviews. While performance audits do not seem to have much effect on budgets they are used frequently to make changes in agencies. One example given related to a port of entry audit which was used to redirect funding from some programs to others that were shown to have a better effect. Mary Noble said that California does not use performance budgeting to any great extent. The State Auditor is required by law to review certain programs and sometimes request are made by the budget committees for program reviews. While performance audits do not seem to have much effect on budgets they are used frequently to make changes in agencies. One example given related to a port of entry audit which was used to redirect funding from some programs to others that were shown to have a better effect.

Tips for making audit work more useful included informing budget committees of the status of recommendations, timing projects to reach the legislature and staff so that they have time to read the reports before hearings, offering briefings, and being succinct in reporting results.

Staff Luncheon

Is Welfare Reform Working? How Do We Know?
Three Perspectives on the Evaluation of Welfare Reform"

Description:  Because of nationwide reductions in welfare rolls, it is widely assumed that welfare reform has been a phenomenal success. Has it? What do we learn from various approaches to welfare reform evaluation? What are their relative advantages and disadvantages? Conclusions may vary based on the perspective of the researcher and the methodologies used to address the issue. The panel presents different insights on what we can learn from studies of welfare reform by focusing on research performed (1) by an individual state, (2) by a national research firm, and (3) by an academic institute.

Moderator:

Kirk Jonas, Deputy Director, Joint Legislative Audit and Review Commission, Virginia

Panelists:

Tom Corbett, Associate Director, Institute for Research on Poverty, University of Wisconsin-Madison, Wisconsin

Alan Hershey, Senior Fellow, Mathematica Policy Research, Inc., New Jersey

Wayne Turnage, Chief Analyst, Joint Legislative Audit and Review Commission, Virginia

Presentation:   Focusing on Program Success:  Assessing Welfare Reform from the Perspective of a National Research Organization by Alan Hershey, Senior Fellow, Mathematica Policy Research, Inc., New Jersey

Presentation:  Evaluating Welfare Reform Initiatives for State Legislatures by Wayne Turnage, Chief Analyst, Joint Legislative Audit and Review Commission, Virginia 

NLPES Luncheon, Business Meeting and
Roundtable Discussion on Contracting Out for Performance Audits

Presiding: Gary VanLandingham, Deputy Director, Office of Program Policy Analysis and Government Accountability, Florida

Each legislative program evaluation or audit office has as its primary goal the creation of a quality product in a timely manner. Each of us spends many hours "in the trenches" in pursuit of that lofty goal. Periodically, we all need a 'pat on the back'-a reassuring 'job well done.' NLPES created its award program as a tangible means to provide that positive affirmation of worth that we all deserve at one time or another.

Since 1982, NLPES has recognized the work of at least 19 of its member states by presenting them with excellence in research methods or legislative program evaluation awards. Other states have received certificates of recognition for the impact their work has had within their respective states. Today, we will continue the tradition of recognizing the work of our member states.

Before I present the awards, I must thank the individuals who gave of their time and effort to select the 1999 award recipients. The following individuals served with me on selection subcommittees for this year's awards: Mary Noble, Don Bezruki, Gerald Hoppmann, Lois Sayrs, Mary Stutzman, Rob Krell, Heather Moritz, Gary VanLandingham, Craig Kinton, Max Arinder, and Roger Brooks. As expected, the award decisions were difficult to make because of the quality of this year's 26 submissions. Without further delay, let's recognize this year's award recipients.

The first portion of our awards program will be the presentation of Certificates of Recognition of Impact. Probably the greatest affirmation of our work comes when legislators or agency personnel utilize information and recommendations contained in our reports to improve government. In an effort to highlight such efforts, NLPES created the certificate of recognition of impact. Today, we will present certificates to eleven offices and highlight the reports that resulted in improvements within their respective states. As I call the name of each office, please send a representative forward and accept the certificate.

Arizona Office of the Auditor General
William Thompson, Director, Performance Audit Division
Performance Audit of the Arizona State Land Department: This report addresses ways to maximize revenues generated from state trust lands administered by the State Land Department, particularly those lands located in urban areas. The report recommended that the Department adopt a more proactive approach.

Florida Office of Program Policy Analysis and Government Accountability
John Turcotte, Executive Director
Performance Audit of The Division of Safety, Florida Department of Labor and Employment Security: This report found that the Florida State Consultation Services Unit was not complying with state law, was not effectively using state assets, and was not significantly impacting private sector safety. Among other things, the report led to reorganization within the Department that included the elimination of 34 full-time staff positions.

Georgia Department of Audits
Paul Bernard, Director, Performance Audit Operations Division
Program Evaluation of the Drug Rebate Program: This report found that the Georgia Drug Rebate program's billing system produced invoices that included large and obvious errors, and that its accounting systems produced records that were untrustworthy. The Department of Audits estimates that average quarterly collections have risen over $5.6 million (41%) following implementation of its recommendations.

Illinois Office of the Auditor General
William Holland, Auditor General
Management Audit of Tuition and Fee Waivers: This report found that in 1995-96, nearly 41,000 tuition waivers worth $117 million were granted. The report received extensive newspaper coverage, and legislation to implement audit recommendations is pending. Full implementation of the audit's recommendations would result in savings of more than $8 million at just one university.

Massachusetts House Post Audit and Oversight Bureau
James Tansey, Director
Report on the Massachusetts Department of Mental Retardation Including "Are You Sure About This Guy?": This report found the provision of care by the Massachusetts Department of Mental Retardation to be substantially impaired. The report received significant media attention. Recommendations for program improvement have already been implemented.

Michigan Office of the Auditor General
Thomas McTavish, Auditor General
Performance Audit of the Charter Schools Office and Michigan Resource Center for Charter Schools, Central Michigan University: This report found numerous problems in the oversight of charter schools by Central Michigan University. The audit received substantial public recognition, including being cited in a report on ABC's Nightline. Numerous operational and programmatic changes have been made as a result of the audit.

Minnesota Office of the Legislative Auditor
Jim Noble, Legislative Auditor
Child Protective Services: This report found that Minnesota's counties have widely varying practices regarding child protective services, that there is confusion about requirements related to the reporting of suspected maltreatment, and that many child care professionals lack confidence in the system's ability to intervene effectively in children's lives. The report received substantial media coverage, and the legislature has passed laws during the last two years to enact many of the report's key recommendations.

South Carolina Legislative Audit Council
George Schroeder, Director
A Management Review of the Department of Natural Resources: This audit found that, although funds of the Department of Natural Resources were used in accordance with the purposes expressed in law, financial management needed improvement. The audit found that some department-wide problems might be avoided by establishing an internal audit department. The report contained 55 recommendations to improve operations at the Department.

Texas Office of the State Auditor
Lawrence Alwin, State Auditor
A Review of Oversight for the State's Embedded Systems Year 2000 Repair Efforts: This report found that many Texas entities did not have contingency plans in the event their computer embedded systems failed, funding was a problem, and no single entity was responsible for coordinating and reporting on statewide repair efforts. The project significantly increased awareness of the problem, and led many agencies to implement processes to help ensure Year 2000 preparedness.

Wisconsin Legislative Audit Bureau
Janice Mueller, State Auditor
An Evaluation of the Petroleum Environmental Cleanup Fund: This evaluation included recommendations to improve the efficiency and effectiveness of the Wisconsin Petroleum Environmental Cleanup Fund program, and to increase monitoring. The report received wide media attention, and a bill to implement many of its recommendations has been voted out of committee and is currently pending action in the Senate.

Wyoming Legislative Services Office
Barbara Rogers, Program Evaluation Manager
Crime Victim Services: This review found that the existing structure of the Wyoming crime victim services, which was split between two entities, was not effective. The review recommended that the legislature consider consolidating the state level administration of victim services. In response, a bill requiring consolidation passed in 1998, with an immediate effective date.

The second portion of our awards presentation is a special recognition award. The Awards Selection Committee was especially impressed with the uniqueness and innovation of one submission and believed that it should be recognized in its own right with a special award.

NLPES presents a 1999 Special Recognition award for accountability reporting to the Florida Office of Program Policy Analysis and Government Accountability.

OPPAGA created the Florida Government Accountability Report, or FGAR, in December 1997. FGAR is a massive web-based encyclopedia that contains profiles that cover each of Florida's 400 major state programs. FGAR is designed to present real time information to legislators who are often unwilling or unable to wait until a traditional program evaluation is conducted before making policy decisions.

FGAR has received many positive comments from legislators and their staff members since its creation. Frequently, legislative staff copy FGAR material directly into bill analyses. New legislators and legislative staff directors have used FGAR profiles to quickly become familiar with the policy areas within their jurisdiction. Since its premier in December 1997, FGAR's tracking system has logged more than 110,000 visits by legislators, their staffs and the public.

NLPES commends OPPAGA for its innovative means of informing the Florida Legislature. I would like to present this award to John Turcotte, OPPAGA Executive Director.

The next award is the Excellence in Legislative Program Evaluation award. The criteria for this award states that the award will be presented to the office producing a printed report or innovative alternative product which utilized exemplary means to provide information to the legislature.

NLPES presents its 1999 Award for Excellence in Program Evaluation to the Washington State Joint Legislative Audit and Review Committee, headed by Thomas Sykes, Legislative Auditor.

The mission of Washington JLARC is to ensure that its work is responsive and useful to the legislature's priorities and issues. To accomplish this mission, JLARC has developed tools to communicate audit results to public policymakers. Realizing that legislators have limited time to read an entire audit report, JLARC develops a one-page report summary for each study. The goal of each summary is to convey the most important information concerning a report in a concise and easy-to-read format. In addition to report summaries, JLARC staff make visual presentations to legislative committees through the use of state-of-the-art presentation technology. Audio recordings of some JLARC presentations can be accessed through an Internet location. In addition to the different means of communicating the results of individual reports, JLARC regularly produces newsletters and summary reports that update legislators on current developments and impacts of JLARC audits and studies. These efforts allow JLARC to summarize concisely the direction, content and results of a wide range of studies, as well as highlight fiscal impacts of some of the agency's major efforts. JLARC's Impacts Report for 1998 states that the agency's 1998 performance audits, program evaluations, sunset reviews and other studies identified possible biennial cost savings of $26 million.

NLPES commends Washington State JLARC for its years of service to the Washington State legislature and its contributions to legislative program evaluation.  Would a representative of the Washington JLARC please come forward to accept this award.

We have one final award to present today.

Like all successful endeavors, NLPES has had a rich heritage of trailblazers, those individuals who have set the pace, those individuals who have tirelessly given of themselves to establish high standards for our profession and organization.

This year, NLPES would like to recognize an individual who has established her own legacy of service to the legislature and has now moved on to a place of service in the executive branch. NLPES presents a special recognition award to Judith E. Frye for her contributions and service to legislative program evaluation and NLPES.

Judy holds a Bachelor of Arts in Political Science from Kent State University and a Master of Arts in Political Science from Michigan State University. Judy began her career in legislative program evaluation in 1979 as an employee of the Wisconsin Legislative Audit Bureau. During most of her nineteen years with the bureau, she was the senior manager of the Program Evaluation Division, which was responsible for performing evaluations and audits of all major and many smaller agencies of state government as well as various local agencies. During her tenure with the bureau, Judy directed projects in the areas of human services, education, transportation, revenue, natural resources, local government, and legislative management. At the time of her resignation from the bureau in 1998, Judy was the Assistant State Auditor for Program Evaluation.

In addition to her service at the Wisconsin Legislative Audit Bureau, Judy has been an integral part of NLPES and NCSL. From 1984 through 1986, she served as NLPES chair and represented NLPES on NCSL's Legislative Staff Coordinating Committee. She also served in leadership positions on NCSL's Fiscal Affairs and Oversight Committee from 1991 through 1993. Judy shared her knowledge of legislative program evaluation by serving as a faculty member of NCSL's Skills Development Training Seminar for more than four years. In 1990, she was a member of the inaugural class of NCSL's Legislative Staff Management Institute held at the University of Minnesota. Judy shared her legislative program evaluation skills internationally by participating in NCSL's 1993 Partnership of Parliaments project with Germany. In November 1998, Judy accepted a position with the Wisconsin Department of Health and Family Services.

It is with deep appreciation that I present this special recognition award to Judith E. Frye for her many, many contributions to legislative program evaluation and NLPES. Would Judy please come forward to accept this award?

"New and Improved Evaluation Products"

Description:  As government becomes more technical and less comprehensible, policy makers look toward performance auditors and program evaluators to demystify state-run programs. Legislators often need a quick turnaround on their requests and may need evaluative information that differs from traditional program evaluations. Program evaluators are increasingly asked to fill a "consultant" role. This session will provide information on new approaches, such as quick response auditing, best practices reviews, and a "thematic approach" to connect program evaluation and the budget process.

Moderator:

Gerald W. Hoppmann, Performance Audit Manager, Louisiana Office of the Legislative Auditor, Louisiana

Panelists:

George Ecola, Consultant, USDA Graduate School Government Audit Training Institute, Washington, D.C.

Kimball Peed, Director, Budgetary Responsibility Oversight Committee, Georgia

Kevin Fillion, Deputy Director, Budgetary Responsibility Oversight Committee, Georgia

Jane Fletcher, Lead Analyst for Education, Office of Program Policy Analysis and Government Accountability, Florida

Kim McDougal, Senior Policy Analyst for Education, Office of Program Policy Analysis and Government Accountability, Florida

California (California State Auditor, Bureau of State Audits)

Introduction to the Bureau of State Audits: This publication introduces the reader to the Bureau of State Audits, which includes a description of the staff structure and various work products.

Implementation of State Auditor's Recommendations: Special Reports to Assembly Budget Subcommittees (Health and Human Services, Education Finance, Senate Budget and Fiscal Review, and State Administration): These reports summarize major findings and recommendations from audit and investigative reports that relate to agencies and departments under the purview of various budget subcommittees. They are submitted to the subcommittees before budget hearings.

Staff Contact: Ms. Margarita Fernandez, CPA
Assistant State Auditor
California State Auditor, Bureau of State Audits
(916) 445-0255
Fax: (916) 323-0913
E-mail: MargaritaF@bsa.ca.gov

Colorado (Colorado State Auditor's Office)

Colorado State Auditor Annual Report: This report was reduced to a pamphlet format a few years ago to facilitate accessibility and communication. It includes various information such as the role of the State Auditor, the mission and performance outcomes for the office, and a listing of released audits.

Privatization Assessment Workbook (1997): This workbook provides a dispassionate, systematic process for state agencies (or any government agency) to use when making assessments about whether to privatize a "government" service.: This workbook provides a dispassionate, systematic process for state agencies (or any government agency) to use when making assessments about whether to privatize a "government" service.

Staff Contact: Mr. Larry Gupton
Deputy State Auditor
Colorado State Auditor's Office
200 E. 14th Avenue
Denver, Colorado 80203-2211
(303) 866-2051
Fax: (303) 866-2060
E-mail: Larry.Gupton@state.co.us

Florida (Florida Office of Program Policy Analysis and Government Accountability)

Florida Government Accountability Report (FGAR): This report is a massive Internet encyclopedia that covers all 400 programs in Florida state government. It also provides "accountability ratings" of programs that are participating in the performance based program budget process. FGAR has received several awards and favorable reviews by publications such as State Legislatures and Governing magazines. For additional information and description of FGAR see http://www.oppaga.state.fl.us/government and http://www.oppaga.state.fl.us/reports/fgarguide.html.

Florida Monitor Weekly: This weekly is an electronic newsletter that provides abstracts and links to recent publications and other information. There are over 350 subscribers. For additional information and to review some back issues of the Weekly see http://www.oppaga.state.fl.us/weekly/.

Staff Contact: Mr. John Turcotte
Director
Office of Program Policy Analysis & Government Accountability
111 West Madison Street
Tallahassee, Florida 32399-1475
(850) 488-0021
Fax: (850) 487-3804
E-mail: turcotte.john@mail.oppaga.state.fl.us

Idaho (Idaho Office of Performance Evaluation)

Federal Mandate Review: These reviews examine federal legal requirements made of the state, associated federal funding, and state compliance with these requirements. Unlike performance evaluations, these reviews do not follow audit procedures and do not conclude with findings and recommendations.

Implementation Report: This report provides the status of all recommendations made by the Office of Performance Evaluation. For each recommendation, there are three columns to indicate whether the recommendation has been implemented, is in progress, or has not been implemented. The report also includes a space for textual explanation of progress, as well as summaries of the Office of Performance Evaluation's estimates of the financial impact by each report.

Committee Letters: These letters summarize reports that have been sent to various committees and also provide updates on progress from evaluated agencies. : These letters summarize reports that have been sent to various committees and also provide updates on progress from evaluated agencies.

Background Papers: These papers are brief documents that summarize background information about a given program and the issues that may be of concern. Unlike evaluation reports, they do not conclude with findings and recommendations for improvement. The Oversight Committee uses background papers to determine whether a full evaluation will be completed.

Staff Contact: Ms. Nancy Van Maren
Director
Office of Performance Evaluations
700 W. State St, Suite 10 JR Williams Building
Boise, Idaho 83720-0031
(208) 334-3880
Fax: (208) 344-3871
E-mail: nvanmare@ope.state.id.us

Kansas (Kansas Legislative Division of Post Audit)

100 Hour Report: This is a special form of performance audit, estimated to take 100 hours or less of staff time to complete. These audits are undertaken only on the written direction of the Chairperson of the Legislative Post Audit Committee to the Legislative Post Auditor. These audits are limited in scope and are often used to determine whether a more comprehensive audit is warranted, or when the scope of the problem is unknown. : This is a special form of performance audit, estimated to take 100 hours or less of staff time to complete. These audits are undertaken only on the written direction of the Chairperson of the Legislative Post Audit Committee to the Legislative Post Auditor. These audits are limited in scope and are often used to determine whether a more comprehensive audit is warranted, or when the scope of the problem is unknown.

Audit Flier: This flier is sent out to media in all Kansas cities with populations greater than 5,000 and all legislators when an audit is completed and presented to the Legislative Post Audit Committee. The flier contains a brief summary of the audit and provides staff contact names.: This flier is sent out to media in all Kansas cities with populations greater than 5,000 and all legislators when an audit is completed and presented to the Legislative Post Audit Committee. The flier contains a brief summary of the audit and provides staff contact names.

Brochure for New Legislators: This brochure provides a brief introduction to the Legislative Division of Post Audit and is specifically aimed at new legislators. After each election, the brochure is sent to new members as a quick introduction to various products and services provided by the Legislative Division of Post Audit. : This brochure provides a brief introduction to the Legislative Division of Post Audit and is specifically aimed at new legislators. After each election, the brochure is sent to new members as a quick introduction to various products and services provided by the Legislative Division of Post Audit.

Audit Matrix: This document, based on a methodology pioneered by the U.S. General Accounting Office (GAO), has proven a valuable tool when planning efficient and effective audits. It lays out audit questions, what information is needed to answer those questions, where auditors can obtain needed information, and proposed ways to collect and analyze information. : This document, based on a methodology pioneered by the U.S. General Accounting Office (GAO), has proven a valuable tool when planning efficient and effective audits. It lays out audit questions, what information is needed to answer those questions, where auditors can obtain needed information, and proposed ways to collect and analyze information.

Staff Contact: Mr. Rick Riggs
Administrative Auditor
Kansas Legislative Division of Post Audit
(785) 296-3792
Fax: (785) 296-4482
E-mail: LPA@postaudit.ksleg.state.ks.us

Louisiana (Office of the Legislative Auditor)

Draft Standards for Quick Response Auditing (QRA): These draft standards were written and paraphrased using the 1994 Revision of the Government Auditing Standards. Although performance auditors and program evaluators are being asked to respond to legislative requests more quickly, we still have an obligation to produce credible and accurate reports. Following a tailored set of QRA standards will allow us to continue meeting the changing needs of our legislature and to produce high quality products in a timely manner.: These draft standards were written and paraphrased using the 1994 Revision of the Government Auditing Standards. Although performance auditors and program evaluators are being asked to respond to legislative requests more quickly, we still have an obligation to produce credible and accurate reports. Following a tailored set of QRA standards will allow us to continue meeting the changing needs of our legislature and to produce high quality products in a timely manner.

Staff Contact: Mr. David K. Greer
Assistant Legislative Auditor and Director of Performance Audits
Office of the Legislative Auditor
P.O. Box 94397
Baton Rouge, Louisiana 70804-9397
(225) 339-3800
Fax: (225) 339-3986
E-mail: davidg@lla.state.la.us

Minnesota (Minnesota Office of the Legislative Auditor)

Best Practices Reviews of Local Government Services: Inspired by the British Audit Commission, which conducts national studies in England and Wales to find out what is the "state of the art" in the delivery of local services, the Legislature created this new tool to improve local government services in Minnesota. These reviews focus principally on counties, cities, and townships. Each review examines a specific, well-defined governmental service, develops a list of cost-effective and innovative practices, provides a forum for exchanging information about best practices, and creates a framework for benchmarking future performance.

Staff Contact: Ms. Jody Hauer
Coordinator of Best Practices Reviews
Minnesota Office of the Legislative Auditor
(651) 296-8501
Fax: (651) 296-4712
E-mail: jody.hauer@state.mn.us

Mississippi (Mississippi Joint Legislative Committee on Performance Evaluation and Expenditure Review)

Lexicon of Legislative Oversight Objectives: This document defines the objectives of legislative oversight, and illustrates them with examples of findings and conclusions from past reports. Since requests for services may call for more than one type of objective, this document is intended to foster a common terminology for those concerned with legislative oversight.: This document defines the objectives of legislative oversight, and illustrates them with examples of findings and conclusions from past reports. Since requests for services may call for more than one type of objective, this document is intended to foster a common terminology for those concerned with legislative oversight.

Staff Contact: Mr. James Barber
Assistant Director
Legislative PEER Committee
P.O. Box 1204
Jackson, Mississippi 39215-1204
(601) 359-1226
Fax: (601) 359-1233
E-mail: james.barber@peer.state.ms.us

Texas (Texas Office of the State Auditor)

Internet-Based Self-Assessment Guides: These interactive self-assessment guides utilize the Internet and help agencies identify problems in key areas. Currently, two guides are available: Human Resources and Automation Controls. Each guide has been further broken down into key control sections. Scores are computed for each of these key controls allowing agencies to determine areas of strengths and weaknesses. All responses are stored in a database in the Office and can be used by the Office during project risk assessments. The Office has plans to develop self-assessment guides for additional key control areas such as student financial aid.

Staff Contact: Ms. Mary Goehring
Audit Manager
Texas Office of the State Auditor
(512) 479-4700
E-mail: mlg@sao.state.tx.us

Texas (Texas Sunset Commission)

Self-Evaluation Report Instructions: This electronically transmitted set of questions is designed to obtain basic background and performance information from a state agency before a Sunset evaluation and review. Key areas covered include major functions, history, policymaking structure, funding, organization, guide to programs, performance evaluation, summary of the prior legislative session, and policy issues. Instructions also provide an opportunity for agencies to provide the Sunset Commission with a preview of issues and suggested improvements regarding the agency and its functions.: This electronically transmitted set of questions is designed to obtain basic background and performance information from a state agency before a Sunset evaluation and review. Key areas covered include major functions, history, policymaking structure, funding, organization, guide to programs, performance evaluation, summary of the prior legislative session, and policy issues. Instructions also provide an opportunity for agencies to provide the Sunset Commission with a preview of issues and suggested improvements regarding the agency and its functions.

Staff Contact: Ms. Ginny McKay
Policy Director
Texas Sunset Advisory Commission
(512) 463-1300
Fax: (512) 463-0705
E-mail: ginny.mckay@sunset.state.tx.us

Washington (Washington State Joint Legislative Audit and Review Committee)

Use of Adobe Acrobat: Adobe Acrobat is used to format documents for Webpage placement. This includes full reports, summaries, newsletters, and annual "impact reports." RFPs are also posted using Adobe Acrobat when major performance audits are contracted out.: Adobe Acrobat is used to format documents for Webpage placement. This includes full reports, summaries, newsletters, and annual "impact reports." RFPs are also posted using Adobe Acrobat when major performance audits are contracted out.

Staff Contact: Mr. Rob Krell
Principal Management Auditor
Joint Legislative Audit and Review Committee
P.O. Box 40910
Olympia, Washington 98504
(360) 786-5182
Fax: (360) 786-5180
E-mail: krell_ro@leg.wa.gov

Wisconsin (Wisconsin Legislative Audit Bureau)

Best Practices Reviews: These reviews focus solely on county and municipal activities, whereas program evaluations only evaluate state programs. In addition, reviews are issued separately from standard program evaluations.

Summary of Evaluations: These summaries are "stand-alone" one to two page summaries of standard program evaluations. Reviews are issued separately from bound program evaluations and are intended to provide the reader with a quick read of key findings and proposed solutions.: These summaries are "stand-alone" one to two page summaries of standard program evaluations. Reviews are issued separately from bound program evaluations and are intended to provide the reader with a quick read of key findings and proposed solutions.

Letters to Legislators: These letters are formal responses to inquiries made by individual legislators when standard evaluations are not necessary.

Staff Contact: Mr. Don Bezruki
Director
Performance Evaluation Division
Wisconsin Legislative Audit Bureau
131 W. Wilson St., Suite 402
Madison, WI 53703-3233
(608) 267-9686
Fax: (608) 267-0410
E-mail: Don.Bezruki@legis.state.wi.us

NLPES Executive Committee Meeting

Presiding: Craig Kinton, Audit Director, State Auditor's Office, Texas