• Mission & Governance
• Services Overview
• How to Get Involved
• Standing Committees
• NCSL Foundation

• Issue Areas: A-Z
• State-Federal Relations
• NCSL Standing Committees
• News from D.C. and the States

• About State Legislatures
• Elections, Campaigns & Redistricting
• NCSL's StateConnect Directory
• Web Sites

• Online Guide to NCSL Services
• Staff Sections and Networks
• Meetings & Professional Development
• Staff Directories

• Meetings Calendar
• Online Registration
• Meetings Home Page
• Legislative Summit

• Bookstore
• State Legislatures Magazine
• NCSL's StateConnect Directory

NLPES Site Map

NLPES News

January 1998, No. 69

Inside the News

Chair's Corner
NLPES Executive Committee: Election Applications Sought
1998 NLPES Awards
Arizona Profile
Liaison Review
A View From the States
-Florida: Fishing for Program Performance with FGAR
-Washington: Riding Herd on Contracted Performance Audits
-Missouri: No Auditing Auditors Says Supreme Court
October NLPES Executive Committee Meeting
Views from D.C.
Office Newsmakers
Web Trappings
News from the Net.doc
Mississippi NLPES Fall Training Conference
Conference Session Notes on the Web
From the Editor
NLPES Calendar

Chair's Corner

Mary Noble (CA)

Before I get into some of the "happenings" in NLPES, I want to thank Max Arinder and his staff in Mississippi for the terrific fall conference they delivered. Not only was the training excellent, but as promised, the hospitality was the best.

The Executive Committee lost a hard working member this month. Craig Monson (UT) has taken a position with the Utah Social Services Department in the child welfare area. Thanks, Craig, for all your good work and your cheerful approach to life. We will miss you. We welcome James Barber (MS) to the executive committee to fill Craig's remaining term. He will be serving on the annual meeting committee and probably many others.

As discussed elsewhere in this newsletter, we now have most of the key contacts on the NLPES list server and Craig Kinton and Bob Boerner continue to update and refine the NLPES home page. Numerous people have told me how willing NLPES members are to share information with each other. Being linked electronically makes it that much easier, especially for those on opposite coasts. I encourage you who have not signed up for the list server to do so. Your work life will be much easier. Just recently Sharon Patnode, the chair of the annual meeting committee, used the list server to ask for input on training. Please make sure you provide it so we can have the training you want!

I wish you all the best in 1998.

Back to the top

NLPES Executive Committee

Election Applications Sought

Deadline: May 1, 1998

Wanted: Six new members for 1998-2000.

Why: A unique opportunity to improve program evaluation nationally through NLPES programs and activities.

What: Travel to three meetings annually, plus your time and your agency's support.

How: Send a brief bio and your goals and proposals on program evaluation (less than 200 words).

Info: Jane Thesing, S.C. Legislative Audit Council, (803) 253-7612; fax: -7639

Send to: Bob Boerner, NCSL, (303) 364-7700; fax: -863-8003

By Mail: 1560 Broadway, Suite 700, Denver, CO 80202

Back to the top

1998 NLPES Awards

Gary VanLandingham (FL)

It is once again time to start thinking about the NLPES awards for outstanding contributions to the field of legislative program evaluation. These awards will be given at the 1998 NCSL conference in Las Vegas. NLPES will offer awards in the same three categories as in 1997:

Excellence in Program Evaluation;
Excellence in Research Methodology; and
Excellence in Impact.

The criteria for each of the awards will be the same as used in 1997.

• The Award for Excellence in Program Evaluation recognizes a state's history of excellence in program evaluation or performance auditing, contributions to these fields, and demonstrated impact on legislative decision making.
• The Award for Excellence in Research Methodology recognizes a state's outstanding use of methods in a research project that was published in calendar year 1997.
• The Impact Award recognizes states that have issued reports that have had significant impact, such as dollar savings, program improvements, or increased legislative awareness. Although the Excellence in Program Evaluation and Excellence in Research Methodology awards are generally given to individual states, there is no limit on the number of states that may receive impact awards.

More information on awards applications and the deadlines will be available shortly on the NLPES homepage and will be sent to NLPES Key Contacts.

If you have questions, please call Gary VanLandingham at (850) 487-0578.

Back to the top

Arizona Profile
Office of the Auditor General

The Performance Audit Division of the Office of the Auditor General was established in 1978 to conduct performance and sunset audits of state agencies. The division also conducts outcome evaluations of selected state programs.

Governing Body: Joint Legislative Audit Committee.

Number and Background of Staff: The division has 49 staff. Staff backgrounds are wide-ranging,including public administration, political science, criminal justice, accounting, social work, statistics, sociology, business administration and economics. Most staff have master's degrees, three have Ph.D.s, and two are attorneys.

We are a cosmopolitan office with staff hailing from all over the country including Alabama, Florida, Tennessee, Minnesota, New Jersey, Colorado, California, Wisconsin, North Dakota, Utah, Illinois, South Carolina, Pennsylvania, New York, Nebraska, Michigan, Iowa and even Arizona!

Most Important Division Asset: The capability, expertise, experience and background of the staff.

Least Important Division Asset: Leftover paper plates from the division picnic.

Recent Audits: Department of Transportation, Universities, Child Protective Services, and the Lottery. Responsible for approximately 45 audits and four program evaluations every two years.

Hot Topics: Welfare reform, LUST (Leaking Underground Storage Tanks), probation, and student enrollment in universities.

Least Interesting Topics for Staff: How we audited back in the 1980s. What it was like to lug around the first generation of personal computers. Anything involving site visits to Yuma.

Longest-Term Office Policy That Never Was: Cannot cook popcorn in the microwave because the odor was unprofessional.

Interesting Audit Experiences: Auditor staying in the same hotel room where a murder had occurred. During a prison audit tour, the same staff person sat in the gas chamber chair used to execute the killer. Air sickness while accompanying helicopter count of desert bighorn sheep. Accompanying state inspectors on a slaughterhouse inspection. Involved in a gun battle with multiple criminals during a State Patrol ride-along.

Equipment Needs: Lie detectors for exit conferences and hearings. Reading glasses for reporters so as not to misconstrue what we say in audit reports.

Most Common Misidentification: We're from the attorney general's office or the Department of Revenue.

Interesting Audit Exchanges: From agency: "This audit will mean the demise of the Materials Division (ADOT)."

If We Could Wish for One Thing: Agency data systems that captured complete and accurate performance-related information. Or, covered parking and cubicles for each person.

Things About Arizona: Dry heat; saguaro cactus; one of the most urban states in the country (85% of the population lives in the Phoenix and Tucson metropolitan areas); the Grand Canyon; state tie is the Bolo tie; and governors unable to complete their full terms of office.

Best Things About Our Office: Excellent staff working to improve state government. Videos shown at annual Christmas party lampooning the year's audit events.

Worst Thing About Our Office: Men not allowed to wear official state tie and women required to wear pantyhose even when it's 120 degrees outside.

Most Common Comment from Reviewing Drafts: It's all there, it just needs a little work!

Back to the top

Liaison Review

National Association of Local Government Auditors

The National Association of Local Government Auditors (NALGA) brings together audit professionals to share common experiences. NALGA is a forum for issues that concern local auditors and a voice for their interests. NALGA provides members an opportunity to discover new ideas and innovations, improve audit quality, and encourage professional standards and ethics.

Established in Philadelphia in the summer of 1989, NALGA is open to everyone. Full members are auditors from local governments, school districts, boards and authorities that conduct audits as a primary duty. Federal and state auditors, and others, may join as associate, nonvoting members. NALGA's annual conference, designed specifically for local government auditors, features recognized experts and practitioners and provides opportunities for peer interaction. The Board of Directors has also initiated projects to improve audit quality, such as a peer review system for local auditors interested in external quality control review, and an automated data base of audit reports, unique audit methodologies and innovative recommendations.

Publications:

• Quality Control Review Guide (1995). A guide to assist auditors in meeting the General Accounting
• Office standards for peer reviews. It also can be used by audit organizations for an internal review to ensure compliance with Government Auditing Standards. The guide is available from NALGA Peer
• Review Committee. Contact Doug Norman of the City of Portland at (503) 823-3542; FAX: (503) 823-4459.
• Guidelines and Model Authorizing Legislation for Local Government Audit Functions (1992). A comprehensive set of guidelines and model legislation for local government auditing. The guidelines address organization of the audit function to make the greatest possible contribution to economical, efficient and effective government that is conducted in accordance with legislative and management directives.
• Local Government Auditing Quarterly. The quarterly contains timely information on activities of NALGA, training, and successful audits from other governments. Membership in NALGA includes a subscription to the quarterly. Nonmembers are also welcome to subscribe.

Current Activities:

• 1998 Conference Planning: NALGA will hold its annual conference in Chicago on May 18 - 19, 1998 -- just prior to the Biennial Audit Forum. The conference committee is developing a strong program in hopes of exceeding the turnout of 180 attendees at the 1997 meeting.
• Knighton Award: Each year at the annual conference the best audit by a member organization is recognized. The award is named for Lennis Knighton -- a pioneer in government performance audits.
• Task Force on NALGA's Future: In 1989 NALGA was founded with 80 members. As we approach 500 members, the board has decided that it is time to look at our future. The task force will review all aspects of membership structure and services. One issue of interest is the question of how to forge stronger ties to the growing number of international members.
• Benchmarking Survey: Our original benchmarking and best practices survey was determined to be valuable in spite of a low response rate. The second version will be streamlined to encourage greater participation.
• Homepage: We are exploring new ways to make the NALGA home page more accessible and useful to members.
• Education: A draft report on local government performance auditing is on its way to the printer. A brochure for elected officials will follow.
• Audit Standards: The NALGA Board has been briefed by the General Accounting Office on pending revisions to the Yellow Book. NALGA will provide input on this important project.

Back to the top

Florida: Fishing for Program Performance with FGAR
Washington: Riding Herd on Contracted Performance Audits
Missouri: No Auditing Auditors Says Supreme Court

FGAR: Electronic Tool for Program Evaluation

Gena Wade (FL)

The Florida Government Accountability Report (FGAR) is a new evaluation product addressing one of the primary challenges facing legislators and decision makers: obtaining current, comprehensive, user-friendly information on the performance of state government programs. This Internet reporting service offered by the Florida Legislature's Office of Program Policy Analysis and Government Accountability (OPPAGA) has just become available, but it is already helping OPPAGA learn more efficient ways of collecting and presenting information that will ultimately improve our other services to the legislature.

The FGAR Web site offers descriptive and evaluative information on 380 state programs. Each program profile is described by legal authority, missions, activities and clients served. Also immediately accessible are OPPAGA reviews, executive branch internal audit reports, auditor general financial and compliance reports, and other assessments of program performance. Profiles contain performance-based budgeting measures tailored to each agency through Florida's statewide initiative, as well as a list of quality-of-life benchmarks to which government programs may contribute across the state. All profiles contain hyperlinks to the Florida statutes, agency Web sites, and other sites of interest.

FGAR's technical design is unique among state government Web sites in that the information is stored in a data base that allows users to search for profiles by agency, policy area, key words, programmatic funding levels, and other characteristics. A variety of search methods are available at the click of a button, and search results also link to related profiles of interest. This search system will allow legislators and researchers to find all the government service and funding streams that may serve particular client populations. Thus, FGAR's design offers an overarching view of government in a way performed by no other research tool to date.

OPPAGA has been committed to this endeavor for other reasons. We have seen increasing legislative interest in enhancing accountability to the public, particularly through performance-based program budgeting. Improving public access to government performance information is consistent with this legislative initiative. Also, term limits in Florida create a need for an objective handbook for new legislators that offers a government-wide perspective.

FGAR meets these interests and serves as the basis for further OPPAGA research in specific programs. Because OPPAGA staff work to become specialists in topic areas, keeping FGAR profiles current encourages staff to stay abreast of policy and performance developments.

Implementation of FGAR took approximately a year and a half, from refining the initial concept and work processes to profiling research and writing and developing the software system, with assistance by Infinity Software Development, Inc. OPPAGA staff will maintain the content and functions of FGAR profiles as part of their research responsibilities.

FGAR will evolve over the coming months, especially with the addition of summary ratings of government programs' accountability systems. Accountability systems provide information decision makers can use for policy decisions and wise allocation of public resources. The system will rate the quality of key components of a program, including: the consonance of its goals and objectives with its statutory mission, performance measures and quality of the supporting data, results reporting, and management use of this information to improve performance. Our rating will inform the Legislature of the quality and reliability of agency information for policy decisions.

Developing FGAR has been a challenging and sometimes exhausting experience, but we are already reaping benefits in its support of our other projects. Our legislators' use of FGAR will be the greatest reward.

Visit FGAR. For comments, contact Gena Wade, FGAR Coordinator or call (850) 487-9245.

Contracting Out Performance Audits

Rob Krell (WA)

Typically, Washington state's Joint Legislative Audit and Review Committee (JLARC) conducts performance audits in-house, using contractors, if at all, to work on certain portions of an audit. In the last legislative session, however, JLARC received two separate mandates to contract out entire performance audits. In both instances, JLARC staff will "manage" the contracts, and the audits themselves will be subject to all of JLARC's standard oversight and reporting processes.

In the first instance, JLARC was appropriated $1.5 million to contract for a performance audit (or audits) of agencies funded in the transportation budget. Following a nationwide RFP process, JLARC awarded one contract to a Virginia firm to conduct an audit of the state's ferry system, and a second contract to a Massachusetts firm to conduct an audit of the state Department of Transportation's highways and rail programs. RFPs to conduct performance audits of the State Patrol and the Department of Licensing will be developed and advertised over the next few months.

In the second instance, JLARC was appropriated $593,000 to contract for a performance audit of the state's workers' compensation system. That RFP was advertised in mid-October and a contract awarded in December.

If your agency is contemplating contracting out large scale performance audits, and has not done so before, feel free to contact JLARC staff by e-mail or call (360) 786-5171 to discuss our experiences and "lessons learned."

Legislative Audit of State Auditor's Office Ruled Unconstitutional
(State Auditor vs. Joint Committee on Legislative Research)

The Missouri Supreme Court ruled that the Oversight Division of the Missouri Legislature's Committee on Legislative Research infringed upon the state auditor's constitutional duties in performing "post audits" of all state accounts. Reversing a Circuit Court decision, the Sept. 30 ruling focused on the words "post audit" in the constitutional provisions creating both the State Auditor's Office and the statutory authority for oversight's management and program audits of state agencies. The court did, however, acknowledge that the division has broad information-gathering authority, and that the Committee on Legislative Research may be authorized to investigate and report to the General Assembly.

The state auditor had also argued that audit work papers prepared by staff were confidential and not open to examination by Oversight Division auditors. The circuit court ruled that such work papers were in fact property of the state and were not confidential under any law of the state. However, the Supreme Court did not address this issue because the audit itself was ruled unconstitutional.

The committee has filed a motion for rehearing by the Supreme Court.

Back to the top

Excerpts from the Oct. 8, 1997, Jackson, Miss. meeting (full minutes are available upon request)

Several suggestions were made for conference changes based on the August NCSL meeting in Philadelphia. These included: Scheduling an NLPES session the afternoon of the last conference day should be avoided. The NLPES awards presentation should occur at the beginning of the luncheon meeting to accommodate agency staff and legislators. Also, the presentation of awards should include more explanation of award criteria and some background on winning reports (especially for the new impact award). The NLPES staff lunch should be made more conducive to socializing perhaps through a semi-circle arrangement of tables. The NLPES reception should be held on the first or second conference day. For the reception at the conference in Las Vegas, executive committee members should explain what will be happening that week and provide a pamphlet. Sharon Patnode (KS) will work on the Las Vegas reception details.

The new NCSL legislative staff achievement award ceremony provided insufficient promotion and explanation of the significance of these awards. NLPES Chair Mary Noble (CA) requested Craig Monson (UT) to draft a letter summarizing concerns on the NCSL awards to forward to Ann Williams, NCSL staff chair.

Kathleen Sullivan (MS) reported on two initiatives being undertaken by the Joint Committee:

1. A $175,000 grant to begin evaluation of students (admissions, statewide testing, etc.), &
2. Revision of personnel evaluations.

Kathleen asked the executive committee to identify persons willing to write standards and case examples. The standards relate to how well the districts are using tests and evaluations. Those interested should be familiar with assessment procedures, but not be experts on specific tests.

Membership and Nominations Jane Thesing (SC) reported that, based on feedback from last year, the voting period will be extended by one week from the time ballots are received by the states and votes are submitted to NCSL.

Communication and Newsletter Craig Kinton (TX) requested that e-mail addresses for all key contacts be confirmed and added to the NLPES list server by November. Also, executive committee members will call key contacts to ensure that the newsletter is being received by all NLPES member agencies. The committee decided to send multiple copies of the newsletter to states based on Bob Boerner's discretion.

Other communications news:
The Works-in-Progress file is being updated. The new NLPES home page is now functional. Bob will provide Craig Kinton and Mary Noble with updates on the status of report loading on LEGISNET. Gary VanLandingham (FL) reported that accessing LEGISNET is not a user friendly operation.

Annual Meeting Sharon went over the annual meeting schedule and requested that key contacts be notified by e-mail and newsletter for topic ideas. Sharon will also contact Jennifer Noyes (WI) for topics.

Training Art Heikkila (AZ) reported that the goal for updating the training guide is fall 1998. Craig suggested that offices be requested to submit applications as sites for the fall training conference. The 1998 conference is in Sacramento, but no future site has been selected. The committee reviewed the criteria for site selection, which indicates that the site should be rotated among North, South, East and West regions.

Awards Craig Monson reported that the criteria for the awards were modified by prior committees. The awards committee feels that the current awards process should continue for the NLPES three awards - Excellence, Methodology and Impact.

Back to the top

Views from D.C.

An e-mail note from our anonymous congressional aid, "I can't believe I work in this business!! These are supposedly true quotes from politicians. I recognized a few of them... but never said any..."
"I resent your insinuendoes."
"No man is an Ireland."
"If we don't make some changes, the status quo will remain the same."
"We're going to have the best-educated American people in the world."
"I support efforts to limit the terms of members of Congress, especially members of the House and members of the Senate."
"If Lincoln were alive today, he'd roll over in his grave."
"We do not have censorship. What we have is a limitation on what newspapers can report."
"Candidly, I cannot answer that. The question is too suppository."
"Outside of the killings in Washington, D.C., we have one of the lowest crime rates in the nation."
"Let's jump off that bridge when we come to it."
"To be demeanered like that is an exercise in fertility."
"I deny the allegations, and I defy the alligators."
"If somebody's gonna stab me in the back, I want to be there."
"When you're talking to me, keep your mouth shut."
"Let's do this in one foul swoop."
"I want to thank each and every one of you for having extinguished yourselves in this session."
"We'll run it up the flagpole and see who salutes that booger."
"I would like to take this time to reirritate my remarks."
"The average age of a 7-year-old in this state is 13."
"I hate to confuse myself with the facts."
"We have a permanent plan for the time being."
"Family planning has many misconceptions."
"The people in my district do not want this highway bypass, no matter if it goes through or around the city."
"My knowledge is no match for his ignorance."
"As long as I am in the Senate, there will not be a nuclear suppository in our state."
"These numbers are not my own; they are from someone who knows what he's talking about."
"People planning on getting into serious accidents should have their seat belts on."
"In 1994, Americans stand on the horns of an enema."

Back to the top

Ohio
The Legislative Office of Education Oversight received the "Notable Document Award" from NCSL's Legislative Research Librarians section for the report, Implementation of Head Start in Ohio.

Missouri
New hires: Greg Beck, CPA, as Performance Auditor II; Julia Miller, CPA, as Performance Auditor III.

Utah
Craig Monson (UT), Legislative Audit Manager and long-time NLPES contributor, accepted a position with the Utah Social Services Department, children and family services.

Washington
Joint Legislative Audit and Review Committee hired Rakesh Moran, formerly with the Louisiana Office of the Legislative Auditor.

Position Available
Nevada: Senior Auditor, Nevada Legislative Counsel Bureau, Carson City. $41,547 - 65,141.

The senior auditor position involves conducting and/or assisting with performance audits of Nevada state government. The position requires the successful candidate to have an advanced degree or be a CPA. Applicants should be experienced in governmental audits, knowledgeable in auditing standards and techniques, have strong analytical abilities, and be proficient in both oral and written communications. Starting salary dependent on prior experience. Send cover letter and resume (including salary history) to:

Legislative Auditor
401 S. Carson Street
Carson City, NV 89701-4747

Back to the top

Web Trappings

The NLPES List Server
Keystroke communications continues...

Use of the NLPES list server is reaching the success once envisioned by Max Arinder (MS) when he first pursued an NLPES Internet bulletin board ten years ago. Convenience is certainly a major factor in the rising e-mail traffic of the last few months. Besides the many publication announcements, complete with Web links to full report texts (thanks to Texas and Wisconsin offices for many of these), many members have emboldened by others' successes submitted both specific and general information requests.

This brings us to the point of this column. Let's call it...

List Server Housekeeping Rules

1. Remember that your message is "broadcast" to everyone on the list server. Consider your message as a speech prepared for an audience of 200+ (sans stage fright).
2. Provide all of the information necessary for a response without undue length, e.g., I would like:
1. "recent" (within two years, three years?); or
2. "any"
3. "specific report/general information" or
4. "resource/personal contacts" on timber management.
3. Include your phone number, e-mail address or Web link within the message for ease of response.
4. Most important, when replying to a list server e-mail, consider whether you should broadcast the response or contact the person directly by personal e-mail or phone.

There were more than 50 list server transactions from September through December. Your agency's key contact now has his/her e-mail address registered. How about you? The more use made of this service, the greater our agencies' interdependence and performance can become. If you are interested in enrolling, click here.

Back to the top

Back to the top  

News from the Net.doc

Gary VanLandingham (FL)

The World Wide Web has experienced tremendous growth over the past two years, and this has its pros and cons for program evaluators trying to conduct research on the Web. On the plus side, there is an abundance of information out there on program evaluation and assessment in numerous disciplines and at all levels of government. Getting quick access to this information can be of tremendous help in getting up to speed in a new program area. Unfortunately, this information is (to be diplomatic) of highly variable quality, and good sites are often hard to find due to the increased volume of information posted on the Web and high net traffic.

Specifically, when evaluators use search engines such as Yahoo and Excite to locate Web pages of interest, they often receive information overload. Search results often produce thousands of Web page links for each query, and sorting through all of the links can be quite tedious, especially if the page you are looking for is number 6742 out of 9761. Furthermore, many of us have experienced the frustration of typing in a query such as "Higher Education Assessment," and somehow receiving Web page link to "How to Train German Shepherds", "Rural Schools in the Andes", "College Football Bowl Previews" (wait, that site does look worthwhile) or other irrelevant sites.

To help overcome these problems, Florida's Office of Program Policy Analysis and Government Accountability (OPPAGA) has created an "Other Sites of Interest" Web page that contains links to verifiably good research sites. This page, is organized into 22 topic areas such as agriculture, health, and re-engineering. Each topic area contains numerous links to good sites for policy and evaluation information. In total, the page now contains several hundred links. Although we created this page for our own use, we welcome others to use it as well.

Our goal is that this site will grow into an information hub that can provide links to any area of government. We ask other NLPES members to let us know about other good sites that they have used by e-mailing Steve Smith, or Steven Birnholz. (These addresses are also on the Web page, and messages can be sent from there.) We will gladly add your favorite sites to the page and light a virtual candle in your honor. So the next time you are about to begin doing research on the Web, try out OPPAGA's "Other Sites of Interest" page and hopefully save yourself some time and frustration. NLPES Conference Focused on the Foundations of Legislative Oversight Steve Miller (MS)

Back to the top

The NLPES training conference

...was held in October, in cooperation with the Legal Services Staff Section, to study the philosophical and legal issues of legislative oversight. The two staff sections met in Jackson, Miss. Many of the sessions explored the theoretical basis for the evolution of compliance auditing into program evaluation.

Midge Smith of the University of Maryland delivered the keynote address for the conference, "Evaluators: Truth Seekers, Hucksters and Acrobats." Smith said that truth seekers are true evaluators who issue fair-minded and thorough reports; hucksters are those who pretend to be evaluators, but who cozy up to agencies and issue false reports; and acrobats are evaluators who would be truth seekers, but never have resources or time to issue a real truth-seeking report.

Michael Scriven, a leading thinker on program evaluation and professor at the Claremont University, presented "The Logic of Evaluation." Scriven urged the use of a general logic of evaluation, rather than performance indicators, as the foundation for conducting program evaluations. Scriven also presented a luncheon speech on the future of evaluation.

One plenary session consisted of a panel discussion on the past, present and future of legislative oversight and explored the effects of term limits and other changes on the legislature's desire and ability to conduct oversight. In another plenary, Mississippi Senator John Horhn gave a dramatic presentation of the African-American perspective on American history.

"The opportunity for learning and sharing audit and evaluation concepts and methods was excellent," said Larry Gupton, deputy state auditor for Colorado.

Eight concurrent sessions covered audit and evaluation issues. Topics included "Risky business: high impact systems for identifying risk and improving accountability," "Using active listening to maximize the productivity of interviews," "Meta-evaluation: evaluating our own work," "Improving the utility of reports," "Data analysis: discovering and visually depicting patterns in data," "Developing needs-based criteria for evaluation," and "Surfing the Internet as a research and dissemination tool." A poster session highlighted innovations in program evaluation from nine states.

The Legal Services Staff Section presented four concurrent sessions on related legal topics. These included: "Jurisdiction of legislative committees over quasi-governmental agencies," "Issuance of subpoenas by legislative committees and witness immunity," "Discoverability of auditors' work papers," and "Working on interdisciplinary teams."

"This meeting offered a splendid opportunity for both groups to broaden their professional skills," said Marcia Goodman, director of Legislative Legal Services for the Connecticut Legislature. Members of both staff sections were encouraged to mix and match these concurrent sessions, and many people attended sessions from both areas. Most participants liked the idea of holding a joint meeting involving two NCSL staff sections.

The group enjoyed several evening events as well, including one in which Elvis Presley appeared in quasi personam postmortem. Mississippi Senator Tim Johnson portrayed the King. During their free time, some members participated in a golf tournament, some took a guided walking tour of historic Jackson, and others visited the State Fair.

Back to the top  

1997 Conference Notes

New to NLPES? Want to know some of the more pressing issues facing evaluation today? Interested in who knows what in some specific area? Go to our Conference Notes page. Find out some interesting things! Most recently added to this site are the following:

97 NCSL Conference Notes

• Child Welfare;
• Cost-Saving Audit Approaches; and
• Reigning in Software Costs.

97 NLPES Conference Notes

• Evaluators: Truth Seekers, Hucksters and Acrobats;
• Legislative Oversight: Past, Present, Future;
• Exploring the Foundations of Legislative
  Oversight: New Currents in the Stream of Evaluation;
• Working on Interdisciplinary Teams; and
• Meta-evaluation.

Back to the top

From the Editor

Mark Bucherl (IN)

Busy, busy, busy! Sometimes it feels like H & R Block tax consultants may get more time off during tax season than performance evaluators get all year! My vest pocket theory is that the "truth seeking" (see Midge Smith's MS presentation) undertaken in evaluation is a much more arduous inclination than knowing when a Form 1099 is called for. That's why I want to thank those who took the time to write articles and respond to this issue's request for contributions. Maybe I'll keep a list and thank you each personally at the 1999 NCSL conference to be held in Indianapolis.

Plug: If you came to Indy in 1988 and had a good time, you will be astonished by how much this town has changed to become even better and more fun... I can't wait to help play your host!

In the next issue, with the capable steerage of Nancy Van Maren (ID), the News will be kicking off a "small shops" column. Working in a small shop myself, I'm excited about the prospects of sharing some the issues Nancy has already developed in talking with other smaller evaluation agencies. On the other extreme, look forward to an article highlighting a three-state (FL, TX & MN) consensus on the most important issues in the development of performance-based budgeting.

My gratitude to the usual accomplices in this office: Debra Hollon (scanning and proofreading), and Kristin Breen, who as I write this, is performing staff duties on an NCSL-sponsored trip to Baltimore to consider chemical weapons. Feel free to write.

  Back to the top

July 24-29, 1999.............................Indianapolis, Indiana
July 15-20, 2000.............................Chicago, Illinois
August 11-16, 2001.........................San Antonio, Texas

Back to the top