NLPES News

National Legislative Program Evaluation Society
Winter 2000, No. 75

Inside the News

Chair's Corner
2000 NLPES Awards
A View From the States - Silver Anniversary Evaluation
Small Shop Talk - Contracting A Cry In the (Idaho) Wilderness
Variations On Murphy's Law
Liaison Review
From the Editor
Calendar
Special Notice

Chair's Corner

Craig Kinton (TX)

	I wish all members of NLPES a prosperous and happy New Year and new millennium. The new century promises to offer many advances in the arts, sciences, and, yes, program evaluation, too. These are exciting times! And, I am certain each of your offices will continue to rely upon NLPES for a variety of networking and
professional development opportunities in the years ahead.

This year our Fall Training Conference will be held jointly with the Research and Committee Staff Section (RACSS) in Austin Texas. We have held previous training conferences in conjunction with other staff sections with good results. This years meeting with RACSS should give us another good opportunity for cross-fertilization. Representatives from the Texas State Auditor's Office, the Texas Sunset Review Commission and the University of Texas LBJ School of Public Affairs are currently laying the plans for this year's conference. We are looking forward to putting on a thoughtful conference with fun and exciting social events.

The National Conference of State Legislatures (NCSL) will be holding its Annual Meeting in Chicago, Illinois on July 16 - 20, 2000. Please let me or any member of the NLPES Annual Meeting Committee know of session topics that you would want to include in this year's program. The meetings will occur at the Convention Center and the Navy Pier. Please plan to attend both the conference and the NCSL Annual Meeting.

Gerald Hoppmann (Louisiana) has taken an executive branch job performing program evaluations in Kentucky and therefore has resigned from his position on the Executive Committee. The Executive Committee has selected David Greer from Louisiana to fill the remainder of Gerald Hoppmann's term. Jane Thesing (South Carolina) was selected to take over as secretary. We wish Gerald the best in his new role and appreciate the willingness of David and Jane to step up and assume additional responsibilities.

That reminds me, nominations for new executive committee members will be coming up in March. Serving on the Executive Committee is a great way to expand your networks and to give something back to the profession. If you are interested in serving, start making your plans now to submit your nomination. More information on the process for nominations and elections will be forthcoming.

Finally, this is the final edition of the NLPES News that will be edited by Mark Bucherl, Indiana Legislative Services Agency. Mark has departed the Indiana legislature to work with municipalities as part of the Indiana Association of Cities and Towns. He will be getting paid for the very skills he has donated to us over these past several years. We will greatly miss his professionalism and newsletter production skills and wish him the best of luck with his future.

I am pleased to announce that Mary Goehring of the Texas State Auditor's Office has been selected as the next editor of NLPES News. In addition to Mary's many talents, she brings experience in editing association newsletters such as ours. We look forward to her contributions on our behalf. Please feel free to share with Mary suggestions for content or improvements in the newsletter.

Please remember that in addition to this periodic newsletter, a content-rich Web site that contains research and training resources, and an active listserv, the staff section offers training conference for both beginning and mid-level evaluators. I hope you continue to let me know of ways we can increase the value of your membership in NLPES. I look forward to seeing you in Chicago this summer.

2000 NLPES Awards

Heather Moritz (CO)

The NLPES Awards application process will begin soon. Generally, the process will be the same as in other years except that the dates have been moved back a little so that the selection process can be completed by the end of May. Key dates this year will be:

April 7 Notice sent to member organizations asking for submissions for the various award categories.
May 8 Submissions due.
May 25 (approximately) Selection process finalized, winners chosen.
Late July (NCSL Annual Meeting) Awards presented.

Awards information is posted on the NLPES Web site (e.g., type of awards, past winners, selection criteria).

Start thinking about applying your agency's best work today! If you have any questions, please contact me by phone or email:

Heather Moritz, NLPES Awards Chair
Managing Legislative Auditor
Colorado Office of the State Auditor
200 East 14^th Avenue
Denver, CO 80203
(303) 866-2051
heather.moritz@state.co.us

A View From the States

Silver Anniversary Evaluation
An Interview with George L. Schroeder, Director, South Carolina Legislative Audit Council

Perry Simpson (SC)

WRITER'S NOTE: In 2000, the South Carolina Legislative Audit Council (LAC) will celebrate twenty-five years of service to the General Assembly and the people of South Carolina. George Schroeder is the only director the Audit Council has ever had. In this interview he reflects on his twenty-five years with the LAC.

Q. What brought you to South Carolina?
A. I came to South Carolina because it gave me the opportunity to build an organization from the ground up. Also, the structure of the Legislative Audit Council appealed to me because it offered more stability. The LAC is overseen by a five-member citizen council, which is unique among auditing agencies in the nation. I felt this gave the agency greater independence. Also, the LAC's director is given a four-year term. At the time I was director of the Joint Legislative Program Review and Investigation Committee of the Connecticut General Assembly. The committee is made up of twelve legislators, and the director serves at the pleasure of the committee.

Q. Has the LAC's approach to auditing changed in the last twenty-five years?
A. Yes. During the first ten years the LAC took a more aggressive approach than we do today. We held news conferences and really tried to sell our audits. Partly that was the result of being a new agency and partly that performance auditing was new to South Carolina government. In a way I guess you could say we were trying to prove ourselves. An unintended result of that is we got something of a reputation among state agencies and they became leery of us. But, after twenty-five years, I think we have proven ourselves. We still work with the media but now we take a somewhat more low key approach. The LAC has, in a sense, become part of the institution of government.
Another important change in our operation came when we adopted the "yellow book" standards in the late eighties. As a result, we are peer reviewed every three years, which serves as a form of quality control. There is also a much greater emphasis on staff training now.

Q. Have you seen a great deal of change in your staff?
A. There has been one very obvious change. When I first started here our entire staff was male. Now, the staff is approximately 70% female. I think this reflects how society has changed in the last twenty-five years. The average age of our staff has also increased, which I think also reflects society. Additionally, when we started we had no special academic requirements for our auditors and most had only a bachelor's degree. Today, we won't hire anybody without a master's degree. In addition, we try to have at least one attorney and one CPA on staff. By the way, our CPA position is currently vacant, so if anybody is interested, please contact us.

Q. How has your relationship with the agencies you audit changed over time?
A. The relationship has certainly improved since 1975. In the early years agencies were not as cooperative as they are now. They would deny us access to records. For example, around 1980, our Department of Mental Health denied us access to some of their records. We went back to our requesters and told them our problem and this actually led to our law being amended to give us subpoena authority. Having never been subject to a performance audit, agencies really didn't know what to expect, and they thought that auditors couldn't possibly come to understand their agency's operations thoroughly enough to be able to make useful recommendations. In fairness to the agencies, we also may have been a bit too harsh in the early years.
Today, things have improved. I think there is an atmosphere of mutual respect and professionalism. We recognize that state agencies have a great many demands placed on them, and they understand that the LAC has a role to play in making government more effective and efficient. As an example of how far we have come, there are times nowadays when agency heads will call to bounce ideas off me. In the old days, they wanted to bounce things off of me, but they weren't ideas.

Q. What is your management philosophy?
A. I am a strong believer in participatory management. I think you put an audit team together with an audit manager, and the team should be allowed to run the audit with as little interference from above as possible. These people are professionals and have the training and experience to do their jobs, and that's what they should be allowed to do. The most important key to the LAC's success is hiring. But once you hire someone you have to have the confidence in them to let them do their job. That is one of the reasons we only hire entry level and then promote from within. I believe that if you spend the time and money to develop somebody, you should promote them rather than hire managers from outside of the agency. Also, I have great confidence in my audit managers since I've seen them develop and know their strengths and weaknesses.

Q. What do you consider to be the most unusual thing that has happened in the last twenty-five years?
A. There have been a number of things. One that does stick out involves an audit we did of our Department of Social Services in our early years. After the audit came out, the speaker of the house decided that we had done such a good job that he proposed that I be made the deputy director for the department. Obviously, that would not have worked, and it never actually came to pass, but it was an interesting proposal. Another interesting thing occurred on our very first audit. It was an audit of our Emergency Preparedness Division, and at the time all of our staff were assigned to it. At the entry conference, I started introducing everybody and, because some of the staff were very new, I forgot some of their names. It was somewhat embarrassing. I guess if that ever happens to me again, I'll know it will be time to retire.

Q. What do you see as the future of auditing in the next twenty-five years?
A. There has been some discussion about how, with the advent of Total Quality Management, that there won't really be a need for performance auditing in the future. Quality teams will be able to identify and correct problems in state government. I don't think that is the case, and I think that the legislature and the citizens of this state will always want an independent agency in place to review the operations of state government. I think the Troggs put it best in their seminal work "Wild Thing" when they said "I think I love you, but I want to know for sure."
I also think that we will be doing more electronic data analysis than we have in the past. With the growth in technology there will be so much more data available and I think auditors are going to be called upon to sort through that data and inform the elected officials and the citizens of the state about what is really important.

Q. Finally, how do you think you've changed personally over the last twenty-five years?
A. Perhaps I could just show you. The first picture here is from 1978. It is from our ID badge. My badge number was 007, like James Bond. The second picture is me today.
A lot of people say auditing will age you, but I really don't see it. I actually think I've aged better than Sean Connery.

Small Shop Talk

Contracting: A Cry In the (Idaho) Wilderness

Nancy Van Maren (ID)

The Idaho Office of Performance Evaluations is a small operation consisting of four full-time evaluators, one part-time evaluation specialist, an administrative assistant, and a director. We often use outside resources to help us conduct fieldwork and analysis. Our office uses these resources to: 1) increase the staffing that can be brought to bear on a project to meet report release deadlines; and 2) obtain expertise that is not available on staff. Not surprisingly, we've found our use of outside resources has had both advantages and disadvantages.

When the office was created in 1993, the Idaho Legislature envisioned that contractors would do all evaluations. As a result, only one position - responsible for administering these contracts - was initially authorized. However, legislative leadership changed direction after learning from prospective job candidates that such a set-up was inherently untenable and that a core permanent staff was needed to properly fulfill the function. Consequently, when a director was hired in 1994, a total of four positions were authorized. Two additional evaluator positions were authorized in 1996.

Since the beginning, our office has used temporary staff and contractors to augment in-house resources. Frequently, temporary or contract staff have worked full-time on specific projects. We've also have used outside resources to perform specific tasks, such as entering survey data or gathering information from agency files. By using temporary staff and contractors, we have been able to take on more projects and still meet deadlines imposed by legislative need.

On a number of projects, we have employed contractors with special, project-related expertise. For example, we hired a professional with expertise in school finance to assist with a school busing study and a contractor with expertise in substance abuse prevention to assist with a review of public school prevention programs. In light of the limited areas of expertise inherent in a staff as small as our own, use of experts such as these has allowed us to effectively respond to legislative requests.

Despite these advantages, there are also some disadvantages to relying on outside resources.

Training - Because of the frequency with which we use outside resources, we are constantly training new staff. While the amount of training required differs depending on the experience level of the individuals involved and the tasks assigned, they all require some degree of training.

Oversight - Since the temporary staff and contractors we use are typically new to our line of work and therefore unfamiliar with the documentation standards we follow, they generally require more oversight than would be required of in-house staff.

Writing - Although the temporary staff end contractors we use may be capable writers, adapting to our office's writing style is often a difficult task. We have found that the training and review time required for a contractor to participate in the writing process is often extensive. As a result, report writing - typically the most stressful part of the evaluation process - has generally been left to the permanent staff.

Turnover - One of the less obvious, but important aspects of relying on contractors is their temporary status. Inevitably, after spending significant time training a contractor, the project is completed, the contract expires, and the contractor moves on to other employment. Consequently, we feel a sense of turnover in the loss of a (now) experienced colleague.

Because of the disadvantages associated with use of temporary staff and contractors, all of us would prefer to have a somewhat larger permanent staff and rely less on outside resources. However, for the foreseeable future we will continue to need the assistance of outside resources to do our work. Learning how to fully benefit from contract help is an on going process. We continually assess how best to quickly train contract help to ensure quality work. We have taken a number of steps to try to minimize some of the difficulties encountered in our use of outside resources. For example, we have:

Adopted a simplified work paper form, something we call an "information paper," which allows the contractor to simply record, rather than analyze, information obtained through fieldwork.

Assigned tasks to contractors that closely match their skills, thereby decreasing the need for extensive training and oversight.
Established ongoing or recurring contractual relationships with certain contractors. Given the nature of contract work and the uncertainties in our project assignments, this is not always possible.

Nancy Van Maren is director of the Idaho Office of Performance Evaluations and coordinates articles for the Small Shop Talk column.

Variations On Murphy's Law

1. The Law of Common Sense
Never accept a drink from a urologist.

2. The Law of Reality
Never get into fights with ugly people, they have nothing to lose.

3. The Law of Self Sacrifice
When you starve with a tiger, the tiger starves last.

4. The Law of Volunteering
If you dance with a grizzly bear, you had better let him lead.

5. The Law of Avoiding Oversell
When putting cheese in a mousetrap, always leave room for the mouse.

6. The Law of Motivation
Creativity is great, but plagiarism is faster.

7. Boob's Law
You always find something in the last place you look.

8. Wailer's Law
Nothing is impossible for the man who doesn't have to do it himself.

9. Law of Probable Dispersal
Whatever hits the fan will not be evenly distributed.

10. Law of Volunteer Labor
People are always available for work in the past tense.

11. Conway's Law
In any organization there is one person who knows what is going on. That person must be fired.

12. Iron Law of Distribution
Them that has, gets.

13. Law of Cybernetic Entomology
There is always one more bug.

14. Law of Drunkenness
You can't fall off the floor.

15. Heeler's Law
The first myth of management is that it exists.

16. Osborne's Law
Variables won't; constants aren't.

17. Main's Law
For every action there is an equal and opposite government program.

18. Weinberg's Second Law
If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would have destroyed civilization.

Liaison Review

National State Auditors Association

Barb Hinton (KS)

The National State Auditors Association, a member of NASACT, the National Association of State Auditors, Comptrollers, and Treasurers, has continued to be even more active and involved in the field of program evaluation in 1999:

Joint Audit Project
The NSAA has selected "Water Quality" as the topic for the 2000 joint audit at its meeting in June 1999. The Tennessee Division of State Audit will coordinate the project. So far, 11 states have expressed interest in the project: Illinois, Kansas, Kentucky, Louisiana, Maryland, Michigan, New York, Oregon, Tennessee, Texas, and Wisconsin. These annual projects combine individual audit efforts from participating states into a joint report that gives a better sense of conditions across the country.

Annual Conference
As NSAA members have become more involved in performance auditing in their states, the NSAA annual conference has provided more and more informative sessions on performance auditing and evaluations. The last few conferences have included sessions on such topics as high-impact performance auditing, performance auditing best practices, uses of program evaluation, information systems auditing, and paperless auditing. The next annual conference will be May 31 - June 3 in Traverse City, Michigan.

Web Sites
NSAA makes many of its members' reports available on the Web. Go to www.osc.state.ny.us/nsaa/ and search for audit reports by topic, keyword, or state. For more information on the NSAA and its current activities, go to the NSAA Web site at www.sso.org/nasact/Auditors/auditup.htm

National Association of Local Government Auditors

Helen Niesner (TX)

The National Association of Local Government Auditors welcomes your liaison and communications efforts with associate organizations. Recent discussions within our organization, as we plan to meet our members needs, recognize that we have both unique and shared needs. For those needs which are particular to us, we will need to develop appropriate response strategies. For those needs which are shared with others, we will want to leverage resources with associate organizations who may already be providing relevant services.
We welcome National Legislative Program Evaluation Society members to our annual conference in New Orleans on May 18 and 19, 2000. You may also visit our Web site at nalga.org.

Institute of Internal Auditors

Jim Pellegrini (MT)

There are several things happening at the Institute of Internal Auditors that relate to program evaluation. Most of what the Institute does relates to braining and networking of internal auditors and evaluators. Besides the normal seminar schedule that can be reviewed on the Institute's Web site at www.theiia.org there is the following:

The IIA recently established a Gaming Audit Group as a special IIA membership section. For an annual fee, it is open to all IIA members involved with casinos, lotteries, racetracks, other gaming enterprises, and government regulatory commissions or agencies. The Group offers a quarterly newsletter, and online forum, a directory of members, and an annual conference. The upcoming Conference dates are April 10 - 12, 2000. The fee for the conference is $445 for IIA members and $495 for nonmembers. However, it is being held at the Flamingo Hilton in Las Vegas, Nevada.

The IIA just held its Annual Conference for State and Local Government Auditors. The conference was held November 1 - 2,1999 in Orlando, Florida. According to the IIA the conference for state and local government auditors keeps getting better and better! The conference addressed the newest trends to impact auditors in the public sector and how some of the most advanced audit shops in the country are dealing with them. The conference is held about the same time each year.

The Seminar Department at the IIA is currently considering developing a course for entry-level governmental internal auditors.

Participation in IIA conferences and seminars provides attendees with techniques, often from the private sector, that can be put into practice when they return to their offices. l have found material and discussion very valuable in helping to redesign evaluation processes. The information and techniques have helped in focusing the evaluation work on getting to the point quicker and keeping things on track.

From the Editor

Mark Bucherl (IN)

This is my last newsletter for NLPES. In January I took a job plying my wares in the communications area in Web site development, as well as in researching/writing various publications. I will grow to miss the frequent contact with ardent practitioners in the field of program evaluation and auditing.

I don't have to remind you that you've got a darn tough job. My first impression eleven years ago was, "Wow, how can anyone figure out all the angles of how this [agency program] works and make a positive change?" You know, as I eventually learned, that issue prioritization, focus and teamwork (including a network of agency staff and supportive legislators) are the keys to effective evaluations. Long-term success accompanies a process bent on continually improving the findings, evolving perspective on the politics, crafting pragmatic solutions and, finally, carefully marketing. The results (voicing truths can hurt).

Though my work experience and evaluation in Indiana have lacked luster in recent years, my captivation of the potential of program evaluation has been buoyed by those dozen or so states whose efforts make appraisal of government a logical and vital part of conducting democracy. There will continue to be lots of work for evaluators and auditors, sorting out right and wrong, possible and impossible, ideal and reality. It's a darn hard...and a darn important job. Keep it up! I'm glad you're out there doing it! Many, many thanks to all the contributors who made this newsletter what it is - keep it up (Mary needs your support)! I wish you well always.

Calendar

March 31 - April 20	Assembly on State Issues Spring Meeting Denver, CO Adam's Mark Hotel Contact: Vicki Rodriguez, NCSL
May 4 - 6	Assembly on Federal Issue Spring Meeting Washington, D.C. Capitol Hill Hyatt Regency
July 16 - 20	NCSL Annual Meeting, Chicago, IL Navy Pier Convention Center and Sheraton Chicago Hotel Contact: Carol Ross, NCSL
July 31 - August 11	Legislative Staff Management Institute, Minneapolis, MN Hubert H. Humphrey Institute of Pacific Affairs Contact: Karl Kurtz, NCSL
August (dates to be determined)	Skill Development Seminar, Madison, WI Contact: Bruce Feustel, NCSL
September 8 - 11	NLPES Fall Training Conference, Austin, TX Omni Hotel Contact: Craig Kinton, TX State Auditor's Office

Special Notice

Rich Jones, Director of Legislative Programs for NCSL, lost his wife Beverly recently to cancer. On behalf of NLPES, the executive committee expresses our sincere condolences and support for Rich and his children Matt and Christine.

Donations may be sent to the Beverly A. Jones Memorial Fund, Norwest Bank Colorado, N.A. Attention: Lee Davenport, 1740 Broadway, Denver, Colorado, 80274, Account Number 0408063576.