NLPES News

January 1997, No.66

Inside the News

Chair's Corner
Assembly on State Issues
Executive Committee Meeting Highlights
KANSAS PROFILE
Liaison Review: American Evaluation Association
The NLPES/AEA Project
Annual Training Conference: A Review and Notes
A View from the States: Open Records and Privatization in Kentucky
NLPES on the Web
From the Editor
NLPES Calendar
Abe Lincoln on Report Writing
Take Home Notes: 1996 Conference Session Summaries

Now that the NLPES Annual Training Conference has come and gone, you may be under the impression that the activities of NLPES and the Executive Committee are more or less over for the year. Not true! A lot of activity takes place within NLPES throughout the year. One major project currently underway is the production of a volume on legislative program evaluation for the American Evaluation Association's publication New Directions for Program Evaluation. A second project that has yet to begin is the development of a casebook to supplement the Joint Committee on Standards for Educational Evaluations' Program Evaluation Standards. A third project that is nearing completion is the publication of an updated Shared Training Handbook which reflects the useful training and materials developed and offered by our member offices.

Several other activities are also ongoing, including planning for the NCSL Annual Meeting and for the next NLPES Annual Training Conference. Efforts are also being made to enhance the NLPES home page on the Web and improve upon the information available to all of our members through the Internet. In addition, as soon as one issue of our newsletter is produced, the production of another begins.

Why am I drawing your attention to these projects and activities? First, I believe that they all present opportunities for you to increase your involvement in NLPES. Although responsibility for each project has been assigned to an Executive Committee member, increased participation and input from others in our diverse NLPES membership can serve to improve the final outcome of our efforts. Second, some of the projects, particularly those involving outside organizations such as the American Evaluation Association and the Joint Committee on Standards for Educational Evaluation, present excellent opportunities for our members to share their knowledge with others as well as to establish some professional publishing "credentials." Third, and perhaps most important, I believe these projects reflect the NLPES commitment to increasing the efficiency and effectiveness of members' day-to-day work. For example, to the extent that we can enhance our ability to share information through such vehicles as the Shared Training Handbook, the NLPES home page, and the NLPES electronic mail discussion group, the greater the likelihood that we will not "reinvent the wheel" already developed by another state.

If you are interested in finding out more about any of the projects currently underway or sharing your ideas about any new projects you think NLPES should pursue, please do not hesitate to contact me or any other member of the Executive Committee. We will meet again in San Francisco on Friday, April 4, in conjunction with the spring meeting of the NCSL Assembly on State Issues.

On another note, on behalf of the Executive Committee, I would like to thank Virginia's Susan Massart for her years of service to the Committee. Susan is replaced by Sharon Patnode, senior auditor, Legislative Division of Post Audit, Kansas.

Finally, I'd like to encourage your participation in the next meeting of the NCSL Assembly on State Issues, described below. The next ASI meeting will be held April 4-6 in San Francisco.

Back to the top

Assembly on State Issues

Jennifer Noyes (WI)

The Assembly on State Issues (ASI) offers NLPES members another chance to exchange ideas and information with other legislative staff as well as state legislators. Although you may be aware of this opportunity, you may question whether your participation in ASI is welcome. The following will hopefully provide the answers. I encourage your participation in ASI meetings and its activities.

What Is ASI?
ASI is one of two committee structures used by NCSL to serve its members. (The other is the Assembly on Federal Issues, which directs NCSL's lobbying efforts in Washington, D.C. and is open only to legislators.)

ASI serves as a major forum for the exchange of ideas and information among state legislatures. It allows each legislator to benefit from the experiences of other states in shaping public policy, experimenting with new laws, and managing the legislative institution.

How does ASI accomplish the goal of sharing ideas and information?
ASI holds three meetings each year, providing an exchange of information through timely and relevant work products and through networking opportunities for legislators and legislative staff.

How is ASI structured?
Nine ASI committees focus on these topics: Arts and Tourism; Children, Families, and Health; Communications and Information Policy; Commerce, Labor, and Economic Development (pending); Criminal Justice; Education; Fiscal, Oversight, and Intergovernmental Affairs; Legislative Effectiveness; and Science, Energy, and Environmental Resources. Two task forces deal with Reapportionment Economic Incentives.

Do I have to be appointed to a committee or task force to participate in ASI meetings and activities?
No! ASI meetings are open to all legislative staff, regardless of committee membership. Staff interested in the issues addressed by ASI are encouraged to attend.

What can I expect from an ASI meeting?
ASI meetings have a structure similar to that of the NCSL Annual Meeting and the NLPES Annual Training Conference. Concurrent sessions are held on a variety of topics relevant to each committee or task force. For example, at the ASI meeting held in December in Washington, D.C., the Fiscal Affairs, Oversight, and lntergovernmental Committee held sessions addressing the effect of federal funding changes on future state budgets, significant state fiscal issues for 1997, the Securities Amendments of 1996, and federal financing of surface transportation programs. Sessions sponsored by other committees addressed such topics as nonpoint pollution and wetland issues, child care choices, and implementation of the Safe Drinking Water Act. Plenary sessions addressing topics of broad interest, such as federal tax reform, were also held.

I need more information. How can I get it?
Additional information about the ASI is available through NCSL's home page, including information about each of the ASI committees, a list of officers, committee work products, and information about upcoming meetings.

Back to the top

October NLPES Executive Committee Meeting Highlights

Craig Monson, NLPES Secretary (UT)

New Award The Executive Committee has created a new award -- an audit impact award. This award will specifically recognize the impact an audit has had on agency operations. An awards subcommittee is developing criteria to be distributed to NLPES members soon.

NLPES/AEA Project A publication on legislative performance auditing is being prepared with the American Evaluation Association.

Training Guide NCSL has compiled a list of training programs conducted by other states. Performance audit shops can use this guide to compare their training with other states. Contact Bob Boerner .

Back to the top

Kansas Profile

History: Auditing in Kansas dates from adoption of the original 1859 state constitution, which provided for a state auditor. The Auditor's Office was abolished in 1970. The Legislative Division of Post Audit was statutorily created in its place.

Governing Body: The Legislative Post Audit Committee, a bipartisan committee, is made up of five members of each legislative chamber. Of the Senate members, three are appointed by the president of the Senate and two are appointed by the Senate minority leader. Of the five representatives, three are appointed by the speaker of the House and two are appointed by the minority leader.

Number of Staff: 20 audit staff, two support staff.

Audit Work: 20-25 performance audits a year, each taking from about six to 16 weeks to complete. About a half-dozen or so financial or financial-related audits are done each year, with most of the financial audits contracted to public accounting firms.

Nickname: "Legislative Watchdogs."

Staff Backgrounds: Accounting, agriculture, agronomy, biology, business administration, chemistry, economics, human resource management, journalism, law, planning, political science, psychology, public administration, research design, and social work.

Most Interesting Current Audit: Studying how corporate hog farms dispose of their, uh, waste products.

Most Disconcerting Audit Experience: Asking maximum security inmates (for an audit of prison food service) why they hadn't cleaned their plates.

Best Thing About Kansas: KU Jayhawks.

Worst Thing About Kansas: Images of Dorothy and Toto, et al., on everything.

Annual Budget: $1.4 million.

Biggest Misconceptions: That somehow we audit posts.

Most Frequent Wrong Number: "This is Post Audit; you want the Post Office."

Best Kudo: In a February 1996 study of Kansas legislative operations, an NCSL review team stated: "The Division of Post Audit is performing high-quality work that satisfies the needs of the Kansas Legislature as expressed by its members. Its work is timely, relevant, useful to legislative policy making, and a cost-effective use of State resources. Kansas currently spends less per capita and per legislator for its audit services than other states with no apparent loss of quality. This reflects the sound management and streamlined organization of this agency." We like them, too.

Most Pervasive Staff Delusion: Agencies like us.

Unofficial State Motto: "Kansas: It's Not As Flat As You Think."

Sharon Patnode, Kansas Legislative Post Audit Committee and New Executive Committee Member
The NLPES Executive Committee is pleased to welcome Sharon Patnode. Sharon began her public career in Ohio in 1977 before moving to Minnesota with her husband in 1979. There she "lived in what seemed like every small town in northern and western Minnesota." She worked four years for a housing authority, administering economic development grants for small towns, before joining a regional planning agency for two years. In 1986, Sharon moved to Abilene, Kan. and enrolled at Kansas State University. There she earned a bachelor's degree in political science in 1990. In 1992, she received a master's in public administration from the University of Kansas. In 1991, she became a performance auditor conducting audits of state agencies and educational and judicial institutions. For the last three years, she has supervised audit teams.

Sharon's interest in becoming more involved in NLPES stems from the view that performance auditing will assume a more important future role as state government continues to contract services to the private sector.

Back to the top

Liaison Review

American Evaluation Association
Rakesh Mohan, NLPES-AEA Liaison (LA)

Although NLPES membership spans all 50 states and is more than 1,000 members strong, a large number of program evaluators do not belong to NLPES. Many of them are members of the American Evaluation Association (AEA)--an international organization for professionals involved in the practice or study of evaluation. AEA has more than 2,000 members who are employed at all levels of government, as well as in the private sector, in nonprofit organizations, and in universities and colleges. In the following paragraphs I will introduce you to the AEA mission, membership and activities. Once you get to know them a little, you will realize the importance of cooperation and sharing of information between NLPES and AEA.

AEA describes the field of evaluation as assessing the strengths and weaknesses of programs, policies, personnel, products, and organizations to improve their effectiveness. The mission of AEA is to improve evaluation practices and methods, increase evaluation use, promote evaluation as a profession, and support the contribution of evaluation to the generation of theory and knowledge about effective human action.

To promote sharing of common interests among AEA members, the organization has 29 topical interest groups (TIGS). Examples of these TIGS include State and Local Government, Evaluation Managers and Supervisors, Quantitative Methods, Qualitative Methods, and Theories of Evaluation. AEA members can join up to five TIGS. Regular membership is $60 per year and includes subscriptions to two AEA journals--New Directions for Evaluation (quarterly) and Evaluation Practice (three issues per year).

At the recent AEA annual conference in Atlanta, presentations were made by people who are familiar to the NLPES community--Don Bezruki, Roger Brooks, Gary Henry, Michael Patton, John Turcotte and Nancy Zajano. All together, there were more than 200 sessions to choose from during the three-day conference. Also, numerous pre-conference workshops covered almost all aspects of program evaluation.

For the first time in 1995, AEA co-sponsored an international evaluation conference in Vancouver (Canada). The other sponsors were the Australasian Evaluation Society, the Canadian Evaluation Society, the Central American Evaluation Society, and the European Evaluation Society. More than 1,500 delegates from more than 50 countries attended the conference.

In spite of AEA activities and its contributions to the field of program evaluation, it is known to only a few NLPES members. In a recent survey I conducted of NLPES member agencies, only a handful of members said that they are also AEA members and have attended at least one conference sponsored by each of the organizations in the last five years. There is a perception among many NLPES members that AEA is primarily an academic organization for social scientists. There is some truth to this; the numbers indicate that 41 percent of AEA members come from colleges and universities; 59 percent hold doctorate degrees; and 43 percent have education, psychology, or sociology as their primary discipline.

However, these numbers do not tell the rest of the story. AEA membership is comprised of evaluators who represent a wide spectrum of professional disciplines, educational backgrounds, and types of employers. I can personally attest to AEA's diversity and its usefulness to all types of program evaluators. As a member of both AEA and NLPES, I have found many similarities in our work, and each group has much to offer the other toward making public programs work efficiently and effectively.

If you have questions, please contact me at (504) 339-3836 (phone) or (504) 339-3988 (fax). For more AEA information, call Rita O'Sullivan, Secretary/Treasurer, at (910) 334-4095.

Note: AEA membership information is taken from its Membership Services Report dated May 1996. The NLPES member agency survey was conducted in October 1996.

Back to the top

The NLPES/AEA Project

NLPES is coordinating the preparation of an issue for New Directions for Evaluation, an AEA publication.

Targeted for the summer of 1997, the issue will feature articles explaining the value that legislative program evaluation adds to the overall field of evaluation. In particular, the issue will highlight successful and practical evaluations at the state level. Articles are being prepared on aspects of legislative program evaluation that differentiate our field from most academic or federal government evaluations. Articles under development include the following issues:

The inherent independence of legislative evaluators from clients.
Unrestricted access to information.
The importance of credibility and alternative inquiry in program evaluations for legislators.

Pieces are also being developed summarizing and highlighting exceptional evaluations performed during the past two years.

Individuals interested in developing articles for the New Directions project should e-mail Kirk Jonas of Virginia's Joint Legislative Audit and Review Commission or call (804) 786-1258.

Back to the top

NLPES Annual Training Conference: A Review

Daryl Miller (WI)

On October 9-12 in Madison Wis., about 125 NLPES members joined members of the NCSL's Research and Committee Staff Section for three days of lively and informative sessions on topics that ranged from project management, to using the Internet for research and welfare reform.

After being welcomed by the Senate president of the Wisconsin Legislature, the conference kicked off with a plenary address by Professor Alan Rosenthal, a nationally known author and scholar of state legislatures and legislative staff. Over the next two-and-a-half days, conference participants were able to choose from 21 different sessions designed to enable program evaluators to do their jobs better. In addition to the various working sessions, two additional large plenary sessions were presented. Professor Don Kettl, UW-Madison, spoke about the Devolution Revolution, and Dr. Stuart Robertshaw of the National Association of the Humor Impaired spoke on the importance of humor in the workplace.

Between sessions, participants had time to relax and talk with colleagues from other states during social events, which included a steam train ride through the Wisconsin countryside to enjoy the fall colors as well as a reception and dinner for all participants.

Next year's conference will be hosted by the PEER Committee in Mississippi. Mark your calendars now for what has become the best training conference available for state legislative program evaluators.

More Conference Highlights: Take Home Notes
Mark Bucherl (IN), NLPES News Editor

Unbiased sources rate our conferences very highly. The Madison staffers were outstanding in organizing this affair, and it was plain to me why the July issue of Money Magazine ranked Madison as the 1996 "Best Place to Live in America."

NLPES conferences are designed to provide program evaluators with useful skills and give them the benefit of other states' experiences in reviewing emerging issues. Such was the case with this conference. Networking opportunities are perhaps the most satisfying element of each conference. At NLPES conferences, networking is built on a solid foundation of inviting topics. So, we outlined a few. Considerable, combined effort went into this idea; let me know if you think such a review is useful.

Back to the top

A View from the States

Open Records vs. Privatization: A Kentucky Experience
Kentucky Office for Program Review and Investigations Committee

Joe Fiala, Director

Government's movement toward privatizing services is testing the balance between the private contractor's wish to keep proprietary information secret and the public's right to know how well these services are performed. A program evaluation office, in seeking to ensure the contractor's legitimate use of public funds, may challenge this balance. The Kentucky Legislative Office for Program Review & Investigations recently did just that. Even after three court rulings, no clear resolution has emerged.

In 1994, we began a review of the operations of Kentucky's state park marinas, which are administered by the Parks Department and its umbrella agency, the Kentucky Tourism Cabinet. Private businesses operate state marinas under renewable, non-bid contracts of 20-30 years duration. To evaluate compliance with required financial reporting and contract payment schedules, we asked to review annual audits submitted to Parks, per contract requirements. Both the Tourism Cabinet and Parks refused to provide the information because one marina operator threatened to sue. Our Committee authorized the use of a subpoena. The operator then convinced several others to join in a lawsuit to prevent us from obtaining these documents. The marina company sued the Kentucky Tourism Cabinet and our Committee, arguing that all financial documents submitted to Parks were confidential and proprietary, and that their public release would provide an unfair advantage to competitors.

The court battle began...

Eventually the suit reached the Kentucky Supreme Court. The Court, reversing two lower court rulings, found that marina audit contents were proprietary and confidential, and, therefore, exempt from disclosure under the Kentucky Open Records Act. However, the Court affirmed our Committee's right to obtain the documents under our own authorizing statute. A dissenting justice remarked that it would be unfortunate if government privatization, in seeking to improve efficiency, economy, and delivery of services, "might be tainted by a cloak of secrecy," and that, "it would be unfortunate if, by contracting out a service, information, which had been available to citizens, could be hidden from public scrutiny."

Reactions to the ruling were mixed. Our Committee chairman, Representative Jack Coleman said, "Privatization doesn't remove government officials from accountability for the delivery of services by private contractors, and it does not remove the legislature from its accountability to the public through legislative oversight of state agency programs." Yet, the Kentucky Auditor's Office interpreted the ruling to mean that it could not conduct planned audits of private marinas, since not disclosing this information publicly would violate state law and its professional code.

The ruling raised questions for our office. Although the Supreme Court gave us access to marina audit records, we could not publicly disclose their contents. Meanwhile, marina operators began submitting all financial information to Parks marked confidential. And Parks began stamping "confidential" on all financial documents sent to us, including routine reports. The irony is that the marina financial information did not comply with contract requirements, and the information it did disclose would hardly have been useful to competitors. The two lower court decisions had shared this view, based upon their actual review of the documents. The Supreme Court ruling, however, was not based upon inspection of the documents, but upon the arguments of the litigants.

The question in Kentucky is, "Where do we go from here?" The Auditor of Public Accounts must now wrestle with new reporting decisions and what actions may risk a lawsuit. As a legislative office, with legislative immunity, it is conceivable that our Committee could vote to release information, despite the Court opinion. Certainly, there is plenty of room for interpretation. Our likely course will be to report data from the financial statements, but not the statements themselves.

One positive action did result. The Department of Finance, at our Committee's insistence, enacted statewide regulations requiring all state contracts to include specifications as to the financial information to be reported and requiring a contractor agreement affirming public access to this information. So far, no potential contractors have refused state business because of this new requirement, despite the arguments of marina operators to the contrary.

Back to the top

NLPES on the Web

Our homepage is http://www.ncsl.org/programs/nlpes.

NCSL.ORG...NLPES
Within this site you'll find this newsletter (with color photos), an online version of NLPES Works in Progress (WIPS), and much useful information about NLPES and its activities.

NLPES List Server Discussion Group
An electronic mail discussion group is available to all NLPES members. Those wishing to join must send an e-mail message to: **listserv@ncsl.org**. Leave the subject area blank and write in the message area: Subscribe NLPES-L Firstname Lastname. After confirmation of subscription (usually within 24 hours), subscribers are able to broadcast messages to all other NLPES subscribers by sending an e-mail message to: NLPES-L@ncsl.org. Likewise, NCSL uses NLPES-L to notify member offices when the newsletter or other new NLPES documents are posted on the Web.

Member Postings on the Web
At least half of our member states, and more every day, now access Web postings. NCSL provides space for time-sensitive material for which NLPES News is not appropriate, such as job openings. Send postings to Janet.Randolph@ncsl.org. For more information, contact our NCSL staff person, Bob.Boerner@ncsl.org, or call (303) 364-7700.

Back to the top

From the Editor

Mark Bucherl (IN)

My thanks to Phil for a pithy and timeless piece of report writing advice from our 21st president. I collect notable thoughts from notable people. Send me your favorites, especially if they apply to program evaluation.

I have an ongoing debate with an attorney friend in our office regarding the usefulness of program evaluation. He worked for awhile for state administration in the executive branch and switched to drafting laws in the legislative branch for the last several years. Conditioned by his witness to government in action, he is a consummate pragmatist. He questions how research can accurately represent legislative interest in an unbiased manner and whether it really matters in the long term. In an instant of manufactured metaphor that caused him some pause, I mentioned that program evaluation inflicts an occasional "stab of conscience" regarding the work or mission of state agencies. I await his next rejoinder (and yours).

Thanks to the numerous evaluators who shared their insights on the conference for the Take Home Notes available on the Web. Special appreciation goes to our staff here, Kristin Breen, Deb Hollon, Susan Knose, and Phyllis McCormick, for writing, scanning, and editing assistance, since we're all half-crazed due to session work! Please call me with your comments at (317) 232-9869 or e-mail me. I want your input in making this communication tool more effective.

Back to the top

NLPES Calendar

April 3 -6	NCSL Assembly on State Issues Hotel Nikko, San Francisco, California ASI is a valuable forum for NLPES members. You can request NCSL to send a brochure at (303) 364-7700, or on the NCSL Website.
June 8-10	The Best of the West: Sharing Best Practices National Association of Local Government Auditors Annual Conference Radisson Plaza Hotel, Ft. Worth, Texas Conference highlights will include: Excellent formal and informal opportunities to discuss key issues with a diverse selection of staff and managers, financial and performance auditors. A pre-conference workshop on auditing performance measurement will be given. Gain up to 18 CPEs at a NALGA-member rate of less than $200. Texas hospitality will include a trip to Billy Bob's. Complete program and registration forms will be mailed in February. If you are not a NALGA member and want registration information, contact Jerome Heer, Director of Department of Audit, 907 N. 10th, Courthouse Annex, Room 319, Milwaukee, WI 53233. Fax: (414) 223-1895.
August 6-9	NCSL 1997 Annual Meeting Philadelphia Convention Center, Philadelphia, Pennsylvania For information, contact Linda Worrell or call (303) 364-7700.
October 8-11	NLPES Annual Training Conference: Program Evaluation Utilization Harvey Hotel and Suites, Jackson, Mississippi "Jackson is a wild and crazy place," said Max Arinder, Director of Mississippi's PEER Committee and 1997 meeting host. With "utilization" as the proposed theme, sessions may include: stakeholder identification, evaluator credibility, project scoping, values identification, and report clarity, timeliness and impact. Social events will include a classic southern shrimp boil and blues music. Optional activities include a golf tournament, a catfish dinner, a tour of the historic downtown Jackson, and visits to the Mississippi State Fair, which happens to coincide with the conference. Some adventurous souls may visit the battlefield at Vicksburg or the casinos on the Mississippi River. PEER Committee and 1997 meeting host. With "utilization" as the proposed theme, sessions may include: stakeholder identification, evaluator credibility, project scoping, values identification, and report clarity, timeliness and impact. Social events will include a classic southern shrimp boil and blues music. Optional activities include a golf tournament, a catfish dinner, a tour of the historic downtown Jackson, and visits to the Mississippi State Fair, which happens to coincide with the conference. Some adventurous souls may visit the battlefield at Vicksburg or the casinos on the Mississippi River. NLPES has extended an invitation to NCSL's Legal Services Staff Section (LSSS) to join the meeting. Several LSSS topics related to utilization involve the following: structuring reports so legislators and bill drafters can put audit and evaluation recommendations into law; issuance of subpoenas by legislative committees; discoverability of auditor work papers; witness immunity; and the jurisdiction of legislative committees over quasi-governmental agencies. The LSSS Board discussed this invitation in January, and we should know soon. For more conference information, contact Max or call (601) 359-1226.

Back to the top

Abe Lincoln on Report Writing

A report having much useless language lay on his desk, the work of a Congressional committee regarding a newly devised gun. "I should want a new lease of life to read this through," groaned the President. "Why can't an investigating committee show a grain of common sense? If I send a man to buy a horse for me, I expect him to tell me that horse's points, not how many hairs he has in his tail."

from Abraham Lincoln: The Prairie Years and the War Years, Carl Sandburg (courtesy, Phil Durgin, Pennsylvania Legislative Budget and Finance Committee)

Back to the top

NLPES News

Open Records vs. Privatization: A Kentucky Experience Kentucky Office for Program Review and Investigations Committee

Abe Lincoln on Report Writing

Open Records vs. Privatization: A Kentucky Experience
Kentucky Office for Program Review and Investigations Committee