• Mission & Governance
• Services Overview
• How to Get Involved
• Standing Committees
• NCSL Foundation

• Issue Areas: A-Z
• State-Federal Relations
• NCSL Standing Committees
• News from D.C. and the States

• About State Legislatures
• Elections, Campaigns & Redistricting
• NCSL's StateConnect Directory
• Web Sites

• Online Guide to NCSL Services
• Staff Sections and Networks
• Meetings & Professional Development
• Staff Directories

• Meetings Calendar
• Online Registration
• Meetings Home Page
• Legislative Summit

• Bookstore
• State Legislatures Magazine
• NCSL's StateConnect Directory

NLPES News
National Legislative Program Evaluation Society

Chair's Corner
Introducing Your NLPES Executive Committee
Mary Noble Recognized for Contributions
Utah Receives Excellence in Evaluation Award
Virginia Receives Excellence in Research Award
16 States Receive Impact Certificates
Wisconsin Legislative  Audit Bureau Profile
Small Shop Talk
NLPES Fall Training Conference Highlights
Association of Government Accountants CGFM
Opportunity for You to Shape Two Panels
Office Happenings
Calendar of Events

---

Chair’s Corner

Just as "hope springs eternal" to baseball fans during spring training, it is always exciting for NLPES Executive Committee members to start another "season" each year around this time. Our 12-person committee—three new members and nine veterans—has lots of ideas for things to accomplish during the 2001-02 year. I’m extremely grateful for the opportunity to serve as chair, and I’m really pleased to have Heather Moritz (Colorado) as vice-chair and Martha Carter (Nebraska) as secretary.

One of the most important functions of NLPES is to facilitate information sharing among our members. One way we do this is through conferences and formal training. Recently, many of you attended NLPES-sponsored sessions at the NCSL Annual Meeting in San Antonio and the NLPES Fall Training Conference in Kansas City. (By the way, big thanks to staff in the Kansas and Missouri program evaluation offices for all of their work in pulling off the first "co-hosted" NLPES fall conference!)

Conferences such as these are great ways for NLPES members to share ideas, but we need still more forums if we are to successfully reach our 1,000+ members. NLPES has tried a couple of new approaches recently that I think we need to continue.
For instance, I was delighted with the quality and quantity of responses we received to a questionnaire sent to each NLPES member agency this past summer. The questionnaire asked agencies to identify their "best practices" in several areas, such as topic identification, staff performance appraisal, and helping new staff get "up to speed." It also asked agencies to identify "hot topics" for possible evaluations, and it solicited suggestions for NLPES-sponsored staff training.

The idea for this survey came from the NLPES Emerging Issues Task Force last spring, and the task force set a goal of getting responses from at least 15 to 20 agencies in the initial survey. In fact, we heard from 29 respondents this summer. Interestingly, we gave agencies the option of submitting their responses via a website (rather than by filling out a hard copy)—and all respondents chose to submit their responses electronically!

The end result of this effort is a rich addition to our NLPES website. I would encourage all of you to go to the NLPES website (http://www.ncsl.org/programs/nlpes/survey.htm) to see the posted responses. As you read them, you’ll see that we bring a lot of different approaches to the ways we do our business, and there is a lot we can learn from each other. The Executive Committee intends to develop a similar questionnaire with new questions for 2002—which should give you even more reasons to visit the NLPES website regularly.

Another forum for exchanging ideas is the NLPES "Question of the Month." Every month, we pose a question to members of the NLPES listserve, and we post the responses verbatim on the NLPES website. There have been ten questions posed since the first one in May 2000 (okay, we haven’t always posted a new question every month). During that time, we’ve received responses from more than 50 separate individuals (many have responded on more than one occasion) in 23 different NLPES member agencies.

I think that efforts like the NLPES "best practices" survey and Question of the Month are examples of exactly what the Executive Committee should be doing: finding ways to (1) enhance information sharing among our member agencies, and (2) engage our individual members in dialogue about issues that affect our day-to-day work. We’ll keep looking for new ways to do this.

Before closing, I want to say a special word of thanks to James Barber of Mississippi for his outstanding leadership as NLPES chair during the past year. Many of the things the Executive Committee will work on during the upcoming year are ideas generated during James’s year as chair. Fortunately, James will be taking on a new responsibility—newsletter editing—rather than being put out to pasture. Those of us who know James’s organizational skills first-hand couldn’t be more pleased.

---

Introducing Your NLPES Executive Committee

Joel Alter. Program Evaluation Coordinator, Minnesota Office of the Legislative Auditor. BA, political science and speech (University of Northern Iowa); MA, public affairs (University of Minnesota). 17 years in field. Chair, NLPES Executive Committee

James Barber. Deputy Director, Mississippi Joint Legislative Performance Evaluation and Expenditure Review Committee. BSBA, management (University of Southern Mississippi); MBA, management (University of Southern Mississippi). 23 years in field. Past Chair, NLPES Executive Committee

Gary Brown. Audit Division Administrator, Michigan Office of the Auditor General. BA, accounting (Michigan State University); MPA (Michigan State University). 29 years in field.

Martha Carter. Program Evaluation Analyst, Legislative Program Evaluation Unit. BA, social sciences (University of Redlands). 4 years in field. Secretary, NLPES Executive Committee.

Ethel Detch. Director, Offices of Research and Education Accountability, Tennessee Comptroller of the Treasury. BA, political science (West Virginia University); MPA, (University of Tennessee-Knoxville). 22 years in field.

Shan Hays. Performance Audit Manager, Arizona Office of the Auditor General. BA, psychology (Utah State University); Master of Accountancy, information systems (Arizona State University). 13 years in field.

Ken Levine. Assistant Director, Texas Sunset Advisory Commission. BA, government (University of Texas-Austin); MPA (LBJ School of Public Affairs, University of Texas-Austin). 20 years in field.

Wade Melton. Staff Director, Florida Office of Program Policy Analysis and Government Accountability. BSBA, accounting (University of Florida); CPA. 28 years in field.

Heather Moritz. Managing Legislative Auditor, Colorado Office of the State Auditor. BA, political science (University of Nebraska-Lincoln); MPA (University of Colorado-Denver). 12 years in field. Vice-Chair, NLPES Executive Committee.

Heather Moss. Senior Management Auditor, Washington Joint Legislative Audit and Review Committee. BA, political science (University of Montana); MPA (University of Washington). 5 years in field.

Jane Thesing. Assistant Director, South Carolina Legislative Audit Council. BA, (Earlham College); MA, history (Indiana University); MLS, library science (Indiana University); MS, business administration (University of South Carolina). 16 years in field.

Kate Wade. Program Evaluation Director, Wisconsin Legislative Audit Bureau. BS, economics (University of Wisconsin-Madison); MBA, management (University of Wisconsin-Madison). 10 years in field.

In addition to fulfilling duties of their ‘regular’ jobs and NLPES positions, Executive Committee members have a wide range of abilities and interests, ranging from playing the harp to baking a ‘killer’ blackberry pie, to coaching girls’ softball to speaking fluent Spanish, to farming organically. Past interests of some Executive Committee members have included co-hosting a nightly music and talk radio program, working as a librarian, and producing a short Claymation film. We extend our thanks to each of these members for contributing their time and energy to the work of NLPES.

---

Mary Noble Recognized for Contributions

On July 23, 1998, the NLPES Executive Committee created an award to recognize individuals who have made outstanding contributions to the field of legislative program evaluation and auditing. The award was established to enable NLPES to recognize individuals who had provided great service to NLPES and who were retiring, leaving the field, or at another point in life that calls for recognition and thanks. This year’s recipient of the NLPES Lifetime Achievement Award is Mary P. Noble, retired California State Auditor.

Mary started her professional career at the United States General Accounting office in San Francisco in 1967 where she was the first woman auditor and one of a handful of women in GAO. Mary managed nationwide audits on topics such as housing discrimination. She received a Meritorious Service Award from GAO.
Mary was appointed California State Auditor in January 2000, a position in which she served until the new state auditor, Elaine Howle, was appointed in August. Previously, Mary was the Deputy State Auditor starting in 1993 when the office was reconstituted. From 1980 until 1993, she was the Deputy Auditor General for Performance Audits.

Mary was instrumental in assisting the Bureau’s return to prominence enjoyed by the predecessor, the California Office of the Auditor General. Under Mary’s watch, the Bureau received the NLPES Award for Excellence in 1998. During her career in both offices, Mary managed over 500 audits and a staff of up to 80 people. Probably the most visible audit on a national level under her direction was the Orange County Treasurer’s Office investment practices. Local, state, and federal laws were changed based upon the audit work.

From 1980 until her retirement in November 2000, Mary was an active participant in NLPES. She served on the Executive Committee for three terms and served as chair in 1997-1998. As vice-chair and chair, she served on various legislative staff committees, including strategic planning. She also served on the Legislative Staff Nominating Committee three times. During her 20 years in NLPES, Mary was a presenter and panel member at numerous national conferences and fall training programs.

In her spare time Mary has completed three weeklong bicycle trips to Alaska, Idaho and Montana to raise funds for breast cancer research. For the Alaska ride, she raised over $11,000, thanks to the generosity of many, including members of NLPES and NCSL. Currently, she is trying to rally her neighborhood to prepare for the next earthquake and helping to organize this year’s ride in the California wine country.
Mary is a caring person of highest integrity and a survivor—both professionally and personally. NLPES sincerely appreciates the many contributions that Mary has made to our profession.
 

---

Utah Office of the Legislative Auditor Receives Excellence in Evaluation Award

Utah's submission met the award criteria by meeting a legislative need, communicating quickly and clearly, and projecting credibility and professionalism. The submission also included some unique material that could serve as a prototype for other NLPES offices to use in improving services to their legislatures and state agencies. Specifically, the Office developed a booklet entitled, "Best Practices for Good Management," that was meant to serve as a guideline for state agencies to use in improving their planning, management, and performance measurement processes. It is aimed at helping managers of new programs set up sound core management processes but could easily be used by managers of existing programs who are looking to make improvements in these areas. The booklet was developed in response to a legislative mandate that added proactive efforts to the Office's traditional auditing responsibilities. The booklet was well received by the legislator who sponsored the bill.

Utah's submission also included two recent audit reports. Both reports dealt with sensitive, "hot" topics. One is an example of the Office's traditional audit report and the other is an example of what the Office calls a "formal letter report." The latter reporting format is used for smaller audits that cover only a few issues. The traditional audit report was entitled, "Performance Audit of Tourism Promotion Funding in Utah." This report was both informative and timely, given Utah's participation as a site for the 2002 Olympic Winter Games. The formal letter report was entitled, "Adoption Time Requirements," and was a response to the concerns of foster parents who expressed frustration with delays in the State's adoption process. Both audit reports were well received by the legislature and the public. In addition, the tourism promotion report served as the basis for legislation passed in the 2001 Legislative Session and the adoption time requirements report served as a catalyst for Utah's juvenile courts to begin standardization and other improvements in the state's adoption process.

NLPES congratulates Utah Office of the Legislative Auditor for its service to the Utah legislature and its many contributions to the field of legislative program evaluation.

Contact:

Wayne L. Welsh, Legislative Auditor General
Office of the Legislative Auditor General
130 State Capitol, P.O. Box 140151
Salt Lake City, UT 84114-0151
(801) 538-1033

---

Virginia JLARC Receives Excellence in Research Methods Award

The committee found that Virginia’s report, "The Costs of Raising Children," met the award criteria for exemplary research methods and was most deserving of this year’s award. The 2000 Virginia General Assembly required this study. The purpose of the study was to demonstrate how existing data could be used to estimate household expenditures on children, and how these estimates could be used to evaluate current child support guidelines or to help determine new, alternative child support guidelines.

This study successfully addressed problems with Virginia’s current guidelines using a sound innovative methodology. JLARC staff used regression modeling in three ways that are different from past studies of expenditures on children. The study recommended that policy makers consider two options, or some combination of the two, in assessing child support. JLARC staff also summarized the advantages and disadvantages associated with the two options.

The Virginia Department of Social Services stated "the report provides a very good basis for the Secretary’s 2001 guideline panel to develop a revised guidelines schedule for consideration in the 2002 General Assembly session." Furthermore, child support experts from other states have expressed interest in adopting the approach developed in this study as well.

NLPES congratulates Virginia Joint Legislative Audit and Review Commission for a job well done and for its contributions to the field of legislative program evaluation.

Contact:

Philip A. Leone, Director
Joint Legislative Audit and Review Commission
Suite 100, General Assembly Building, Capitol Square
Richmond, Virginia 23219
(804) 786-1258

---

Sixteen States Receive Certificates of Impact During 2001 NCSL Annual Meeting

Annually, NLPES presents certificates of impact to offices releasing reports that have documented public policy impact on their respective states. During the NLPES Business Luncheon held on August 15, 2001, the following states received such certificates.

Arizona
Department of Public Safety Scientific Analysis Bureau. This audit found that the two units—DNA and toxicology—within the state’s Scientific Analysis Bureau, or crime lab, were experiencing serious backlogs. Contact-Dot Reinhard, 602-553-0333

Colorado
Colorado Preschool Program, Department of Education, Performance Audit. This audit of the state’s preschool program found that the Department of Education had failed to meet its statutory authority to oversee this program. Contact-David Barba, 303-866-2051

Florida
Justification Review, Toll Operations Program, Florida Department of Transportation. This report set out to identify ways to improve the Toll Operations Program’s performance and reduce state costs. Contact-John Turcotte, 850-488-0021

Georgia
Sufficiency of the Hazardous Waste Trust Fund. This report studied the sufficiency of the state’s Hazardous Waste Trust Fund. Contact-Russell Hinton, 404-656-2174

Hawaii
Actuarial Study and Operational Audit of the Hawaii Public Employees Health Fund. The audit found several problems with the board’s management of the Public Employees Health Fund, including failure to ensure that the fund’s reserves were being properly managed, ineffective rate stabilization efforts, and failure to return excess reserves to employees. Contact-Marion Higa, 808-587-0800

Illinois
Department of Public Aid’s Child Support State Disbursement Unit. This audit came about because of drastic problems that occurred when Illinois attempted to implement the federally required child support State Disbursement Unit. Contact-William Holland, 217-785-2982

Kansas
Reviewing the Implementation of the Mental Health Reform Act. This audit reviewed the Department of Social and Rehabilitation Services’ implementation of the Mental Health Reform Act, under which the state’s psychiatric hospital beds were reduced and community mental health services were increased. Contact-Rick Riggs, 785-296-3792

Michigan
Performance and Financial Related Audit of the Child Development and Care Program, Family Independence Agency. The audit found that the Child Development and Care Program often did not comply with federal or state statutes, regulations, rules and policies and procedures for the distribution of child care funds. Contact-Thomas McTavish, 517-334-8050

Minnesota
Fire Services: A Best Practices Review. This report provided information to Minnesota’s 790 local fire departments on effective and efficient fire services. Contact-Jody Hauer, 651-296-8501

Mississippi
The Mississippi Department of Transportation’s Administration of the 1987 Four Lane and Gaming Roads Programs. The report identified causes of the significant cost overruns and delays in two highway construction projects. Contact-James Barber, 601-359-1226

Montana
Microbusiness Finance Program. The report identified significant weaknesses in a program that provided loans to small businesses. Contact-Jim Pellegrini, 406-444-3122

Texas
An Audit Report on the Department of Economic Development. The audit reported on gross fiscal mismanagement in a program in which employers contracted with the state to train employees. Contact-Craig Kinton, 512-936-9360

Utah
A Performance Audit of the Central Utah Water Conservancy District. This audit of a quasi-governmental organization was responsible for the district’s funds being invested for greater return. Contact-Wayne Welsh, 801-538-1033

Virginia
Virginia’s Medicaid Reimbursement to Nursing Facilities. This study found that certain parts of Virginia’s Medicaid reimbursement methodology had not been adequately updated and appeared to be excessively restrictive. Contact-Philip Leone, 804-786-1258

Washington
Mental Health System Performance Audit. The audit found that mental health services to clients were not well coordinated with other human services, that system accountability was burdensome, and inequities in funding led to inequities in services. Contact-Tom Sykes, 360-786-5171

Wyoming
Wyoming State Archives. This report found that despite the explosion in electronic records, the department had maintained its focus on the traditional paper record environment. Contact-Barbara Rogers, 307-777-7881
 

---

Wisconsin Profile 

Employees on the Payroll: 80: The State Auditor and her Deputy (2); Financial Auditors (42); Program Evaluators (26); Administrative and Clerical (6); and Information Systems (4)

Age Distribution of Employees: 34 under the age of 30; 24 who are "thirty-something"; 22 who are over 40…and that’s all that can be said in polite company!

Number of Computers Available for Staff Use: 92 (63 notebooks, 29 desktop PCs). (In 1988, we had 23!)

Cost of a cup of coffee: 35 cents (It was 2 for 25 cents in 1988!)

Adventures in Auditing: Arriving at the hotel up north on a Sunday night in the middle of winter to find a sign that says, "Closed."

Biggest Misconception: That we are all accountants.

Biggest Mistake made while on an audit site visit: Leaving a list of questions with an auditee, who then sent the questions to other offices around the state to "prep" them for our visit.

Biggest Misunderstanding: Audit Bureau vs. Auto Bureau

Common Questions asked by Auditees: So, are you guys still in school? (See age distribution!)

Hot topics: Budget Surplus, Budget Deficit

Curious Concepts in Agency Performance Reports: An agency report showed the number of students served in each district. In many school districts, the number of students served was several times greater than the number of students enrolled in the district. Run that by me again?

Two Dream Assignments: 1) On-site auditing of the Department of Commerce office in Frankfurt, Germany. 2) Auditing the State Fair’s cream puffs; samples would likely need to be drawn.

Great Moments in Wisconsin History: 1854, Birthplace of the Republican Party; 1878, First Ice Cream Sundae created; 1887, William Horlick invents Malted Milk; 1949, Joseph Zimmerman of Milwaukee invents the first telephone answering machine

Two Most Disconcerting Audit Experiences: 1) After a 90 minute interview, having the auditee ask to review our notes. No, we didn’t have another 90 minutes for that purpose. 2) Having a prison as your audit site on your third day of work…and having to take your bra off to make it through the metal detector.

Not for the Faint-hearted: Clubbing 30 trout so that the agency’s fish health specialist could evaluate their organs.

Strange Audit Experience: Watching an Assistant Principal chew out truant kids as part of our review of Truancy Practices.

Unofficial Motto: We’re here to help.

Nickname: Wisconsin’s Watchdogs
 

In a World Where BIGGER is Better, Can Small Shops Be Taken Seriously?

This is, by my count, the sixth installment of "Small Shop Talk" since the inception of the column in May 1998. The topics covered thus far, introductory article aside, have focused on employee turnover, hiring practices, contracting, and the challenges of training. This clearly implies, and correctly so, that the biggest challenge facing small shops is staffing—when there is a big job to be done and only a handful of bodies to do it, success depends, in large part, on whom those bodies belong to.

In this article, I am going to address a different issue that is just as important as staffing to a small shop’s success. I am going to discuss a few things that small shops can do to make their work more relevant and ensure that they are taken more seriously.

What do I mean when I imply that small shops may not be taken seriously? I mean that small shops often do not get the respect they deserve because of their inevitably low output. For example, the Nebraska Program Evaluation Unit has released, on average, one preevaluation inquiry and 2.3 full evaluations per year since 1995. With only four full-time analyst positions available, and given the problems of hiring and training noted above, we are lucky if we have three experienced evaluators working at any given time. Often, we have just two. In terms of productivity per employee, we are doing fine—but the fact remains that programs in Nebraska have about as much chance of being evaluated by us as a person has of being struck by lightning. Nebraska is not unique among small shops in this regard.

This fact is not lost on both legislators and staff members who, as a result of information overload, often default to a form-over-substance view of the world. Regardless of quality, many look for quantity; volume is the thing most likely to get their attention. Because small shops don’t have the staff to produce a large number of reports every year, there is a misperception that they are splashing around in the kiddy pool and aren’t having an impact.

The reason for this misperception is largely organizational. As anyone working in the program evaluation field knows, bureaucracies carry their own momentum. The size of an organization is often used, incorrectly in many circumstances, as a measure of its importance. In Nebraska, it seems that legislators often forget we exist. As a result, when a legislator has a question or concern about a state agency, he or she often overlooks program evaluation as a tool for addressing the issue.

Sometimes, it is even difficult to get the attention of our own committee. Nebraska’s Program Evaluation Committee is a special committee rather than a standing committee, which means that it doesn’t meet regularly, even during the legislative session. Trying to fit meetings into senators’ busy schedules is difficult because we are competing with issues and activities that are more routinely before them. When we do meet, committee members are often rushed as they try to fit the meeting in between other appointments, and the level of engagement is unpredictable.

So what must small shops do to overcome these obstacles and to be taken seriously? There are a few key factors. First, a small shop must seek out allies—legislators and staff members who believe in the program evaluation process—and cultivate those relationships. There is a difference between those who give lip service to the importance of evaluation or think about it only when issues that interest them arise, and those who believe in the process as an integral component of legislative oversight. Networking with those who understand the importance of program evaluation makes it more likely that legislators appointed or elected to oversee the process will be individuals who are interested and engaged.

Second, small shops must be aware of the political environment in which they are working. They should be attentive to developing issues that may lend themselves to program evaluation,
so that the process becomes more relevant to the legislative body. I am in no way suggesting that evaluators sacrifice their objectivity. Objectivity is still their bread and butter—without it, no one will give weight to recommendations. But for reports to be more relevant, at least some of the time they must address issues of significance that are before the legislature.

Finally, reports must be timely. Committee meetings must be scheduled to ensure that action can be taken at the most opportune time. Evaluators must learn to tailor evaluations to tight time frames if it means that recommendations are likely be implemented quickly. An exhaustive report does no good if it recommends legislation but isn’t produced in time for bill introduction. As anyone in the legislative arena knows, interest in issues can wane, and recommendations can become stale before legislation is introduced. This is not always a concern, and thoroughness should not be sacrificed, but small shops should not let opportunities to have an impact slip by.

In conclusion, I’d like to note that the suggestions discussed above are important not only to small shops; they need to be addressed by all program evaluation units. However, they are even more important for small shops because of our often low visibility. When staffing levels and output are low, program evaluation units must take extra care to ensure that their reports are meaningful and timely. It’s not easy, but working harder and smarter is the little guy’s key to success in a world where bigger is better—it isn’t the size of the dog in the fight, it’s the size of the fight in the dog.
 
 
 

---

Fall Training Conference Provides Training and Networking

From September 4-8, more than 134 legislative evaluators and auditors gathered in Kansas City to learn new skills and network with colleagues from across the U.S. Co-hosted by the Missouri Committee on Legislative Research, Oversight Division, and the Kansas Legislative Division of Post Audit, the NLPES fall training conference was a blend of learning opportunities and enjoyable social activities. The following photos capture the atmosphere of the conference.
 
 
 

Harry Hatry, Urban Institute	Catherine Barrett, Governing Magazine
Mark Funkhouser, Kansas City Auditor	Claire McCaskill, Missouri State Auditor
Ramona Kenady, NCSL Staff Chair	Claire, Barb, Melanie, and Joel enjoy lunch
Opening reception networking	More networking and FOOD!
Making new friends	"They never said fieldwork would be like this!"
Cowboy Bob riding " Beelzebob"	Joel congratulates Jeanne and Barb for a job well done

---

NCSL Self-Assessment Tool

Legislatures are also unique institutions, where management principles developed for business or public agencies do not always work well. But there are management principles that contribute consistently to successful staff agency operations.

The NCSL Legislative Staff Coordinating Committee (LSCC) Task Force on Promoting Professional Development for Staff recently released a very useful guide. The "Self-Assessment Guide for Legislative Agencies" (May 2001) provides individuals with a tool to evaluate the effectiveness of their organizations in specific issue areas.

The guide is designed to help legislative agencies compare their management practices with the principles of effective management. The guide is intended to raise questions about current operations and to stimulate thinking about ways to improve staff performance. The principles apply to all types of legislative staff agencies, and managers at all levels from first-line supervisors to agency directors can make use of the guide. Chapters in the publication include "Management Structure," "Communication," "Personnel Management," "Office Automation and Information Systems," "Budget Management and Control," "Physical Plant," "General Assessment of Work Environment" and "Manager's Individual Progress and Performance."

The guide is based on the expertise of experienced and successful staff managers. It was originally developed by the Legislative Staff Coordinating Committee of NCSL in 1991, and revised in 2001 by the Task Force on Promoting Professional Development for Staff. The task force updated the original version to reflect current information technology and reviewed and revised it to meet the unique needs of legislative agencies at the beginning of the twenty-first century.

The guide will be available shortly on the LSCC's Web site at: http://www.ncsl.org/legis/lscc/lscc.htm or you may contact Ron Snell with NCSL at (303) 364-7700 or Ron.Snell@ncsl.org for additional information.
 

---

Opportunity for You to Shape Two Panels

Rakesh Mohan (Rakesh is a Principal Management Auditor with the Washington State Joint Legislative Audit and Review Committee. He also serves as NLPES’s liaison to the American Evaluation Association.)
 

Whether or not you are planning to attend the upcoming annual conference of the American Evaluation Association (AEA), you can shape the discussion at two panel sessions by providing suggestions for questions you would like to have addressed, or issues you would like to have discussed. The conference will be held in St. Louis from November 7–10, 2001, and the two panels are scheduled for November 8.

The panels are on two topics that are very relevant to NLPES members—performance measurement and evaluator certification. Performance measurement has become almost an integral part of state government, and has been a major topic of discussion among members of both NLPES and AEA. On the other hand, the issue of whether there is a need for evaluators to be certified is being discussed primarily among AEA members. Yet it has the potential of affecting all evaluators regardless of their membership in either AEA or NLPES.

I will use your suggestions to develop a framework for facilitating the discussion for both panels. For example, you can ask panel members to:
Address specific issues or answer a nagging question relating to either topic.

• Comment on your particular experience in developing, using, or evaluating performance measures.
• Comment on your examples of certification processes that may work for evaluators.
• Comment on your views, in favor of or against, the idea of certifying evaluators.

See below for details about the panels. Information about the conference can be found at the AEA website (www.eval.org). I look forward to hearing from you.

Panel: Ask the Experts . . . Everything You Wanted to Know about Performance Measurement but Were Afraid to Ask

Panelists: Joseph Wholey (University of Southern California and US General Accounting Office); David Bernstein (Montgomery County, Maryland, and Consultant to Governmental Accounting Standards Board); Ian Davies (University of Victoria, Canada); and, Jules Marquart (Tennessee Department of Children’s Services)

Abstract: This interactive panel of experts from diverse backgrounds will discuss key issues and challenges facing evaluators, program officials, and policymakers about performance measurement. For example: How can useful performance measures be developed? How does one ensure that measures are reliable and valid? Should performance measures be used to punish or reward programs through the budget process? Should performance measures be used to compare and rank "similar" programs in different cities and states? How to develop cost-effective measures?

A sample of pre-selected questions will be used to initiate the discussion among panel members. Audience members will be encouraged to ask questions during the entire 90-minute session.

Panel: Certification of Evaluators: Do We Really Need It?

Panelists: James Altschuld (Ohio State University); Nancy Zajano (Legislative Office of Education Oversight, Ohio); Linda Triplett (Mississippi Legislative PEER Committee); and, Nancy Kingsbury (US General Accounting Office)

Abstract: This interactive panel of professionals from diverse evaluation backgrounds will discuss key issues and challenges relating to the certification of evaluators. In recent years, there has been considerable discussion on the issue of certification for evaluators. The fall 1999 volume of the American Journal of Evaluation has brought the key points of the certification debate to the forefront. Although the journal lists a number of reasons for having some sort of a certification process, it falls short of providing evidence to show that significant harm has been done to the public because the field of evaluation lacked a certification or licensure process. Furthermore, it is not quite clear what would be the impact of certification on the interdisciplinary nature of the evaluation profession.

A sample of pre-selected questions will be used to initiate the discussion among panel members. The panel members hold various views concerning certification, including for, against, and somewhere in between. Based on their extensive evaluation backgrounds, they will be able to facilitate a discussion that will clarify some of the unresolved issues about certification. Audience members will be encouraged to ask questions during the entire 90-minute session.

Association of Government Accountants Promotes the Certified Government Financial Manager (CGFM) Designation

The challenge facing today’s government financial managers is to keep up with changing times. Innovations, developments, and methods of implementation are creating a new environment in which government financial managers must function. Increasing knowledge and skills is the key to staying abreast of these latest developments.

In 1994, the Association of Government Accountants developed the Certified Government Financial Manager (CGFM) designation. Since that time, the CGFM has become the standard by which government financial management professionals are measured. The CGFM’s education, experience, and ethics requirements have served to elevate the most seasoned financial professionals to excellence. More than 13,000 have received the designation to date.

The CGFM consists of three comprehensive examinations which address the following topics: governmental environment; governmental accounting, financial reporting and budgeting; and, governmental financial management and control. Each of the three separate examinations is structured to require a candidate to demonstrate a general familiarity, understanding and appreciation of the subject area.

The CGFM examinations are administered year-round in a multiple-choice, computerized format on behalf of AGA by NetCertification, Inc. via the Sylvan/Prometric Technology Centers. This computer-based administration allows the CGFM examinations to be offered securely in more than 300 locations in the United States, reducing travel costs and allowing candidates to schedule exams at their convenience. To learn more about the examinations, visit www.agacgfm.org
 

---

Office Happenings

Florida

Jeanine King, Michelle Harrison, Nanette Smith, Scott Stake, LucyAnn Walker-Fraser, Rae Hendlin, Steve Stolting, Don Pardue, Sarah Mendonca-McCoy, and Benny McKee are recent hires at OPPAGA.

Daniel Starr and Jose Gonzales completed 12-week summer internships at OPPAGA.

Dick Brand and Sharon Anderson have recently left OPPAGA for other employment opportunities.

The Florida Legislature recently passed the Sharpening the Pencil Act to improve school district management and use of resources and to identify cost savings. OPPAGA is charged with implementing this aggressive review of all school districts on a 5-year cycle. The Legislature appropriated $3.2 million to conduct the reviews. In order to take on this additional workload OPPAGA anticipates hiring staff with school district backgrounds to supplement its existing staff.

Baby OPPAGAn…Rebecca Urbanczyk became the proud mother of daughter, Mary Grace Urbanczyk, on July 10.

Minnesota

Elliot Long recently retired after 24 years with the Program Evaluation Division.

Mississippi

Karen Kerr, Oona McKenzie, Sara Watson, and Candice Whitfield have recently joined the PEER Committee as evaluators.

Kim Haacke, Tanya Webber, and Julie Winkeljohn recently left PEER for other employment opportunities.

PEER staff have recently moved into newly renovated space in the E.T. Woolfolk State Office Building.

South Carolina

Anne Marie Cotty and Susan Poteat, both recent graduates of the University of South Carolina’s Master of Public Administration program, recently joined the Legislative Audit Council.
 

Washington State

Shayne Frost has recently joined Washington JLARC.

Ron Perry and Elizabeth DuBois recently left JLARC for other employment opportunities.

Legislators and staff from Washington’s JLARC were very active in organizing and moderating NLPES panels presented during the NCSL Annual Meeting in San Antonio.

West Virginia

David Mullins has recently joined the Program Evaluation and Research Division as a Research Manager.
 

---

Calendar of Events