• Mission & Governance
• Services Overview
• How to Get Involved
• Standing Committees
• NCSL Foundation

• Issue Areas: A-Z
• State-Federal Relations
• NCSL Standing Committees
• News from D.C. and the States

• About State Legislatures
• Elections, Campaigns & Redistricting
• NCSL's StateConnect Directory
• Web Sites

• Online Guide to NCSL Services
• Staff Sections and Networks
• Meetings & Professional Development
• Staff Directories

• Meetings Calendar
• Online Registration
• Meetings Home Page
• Legislative Summit

• Bookstore
• State Legislatures Magazine
• NCSL's StateConnect Directory

NLPES Site Map

NLPES & RACSS

Joint Fall Training Conference Notes

Track 1: New Work Demands

Excellence Panel
Writing for the Future
New Work Demands: Emerging Issues in Performance Measures in the States
Effective Contracting for Services
Auditing Agencies That Don’t Work: Controlling the Costs of Conflict

Excellence Panel

Moderator: Heather Moritz, Managing Legislative Auditor, State Auditor’s Office, Colorado

Shan Hays, Performance Audit Manager, Office of the Auditor General, Arizona

Ms. Hays described the audit of the Arizona State Hospital, for which her office received the NLPES Excellence in Evaluation award. Criteria for the award include meeting a legislative need, communicating quickly and clearly, projecting professionalism, and assisting other NLPES offices. This presentation was intended to assist other NLPES offices by giving them information in case they wanted to use a similar technique to present their findings.

The audit was not unusual, and fieldwork was a straightforward matter of documenting problems and developing sound recommendations. The office used a combination of a standard written report, publication on its web site, and an 11-minute video presentation to communicate its findings. The video was shown at the legislative hearing and subsequently copies were requested by other legislators, the Governor’s office, and a local public affairs television program. Ms. Hays presented the video in its entirety. The video described the audit findings, which focused on staffing problems, a deteriorating facility, and the need to conduct long term planning, and provided information on the hospital’s Medicare certification and a new state program for sexually violent persons. After the audit, the Governor and Legislature approved spending $80 million for a new hospital, and the hospital implemented all the audit recommendations and succeeded in regaining its Medicare certification this summer.

Advantages of using a video include that it’s easily understood, convenient (compared to an oral presentation), memorable, and attention getting. Offices that want to use videos to present their findings will need equipment and staff resources. Arizona minimizes time and money by using a student intern and a larger state agency’s editing equipment. The script is the critical part of the video process; it details the narrative and accompanying shots. Filming can be made more efficient by finding other sources such as training videos, and shooting footage during the audit; a 10-minute video requires about 100 different shots. Editing takes about 2-3 hours per minute of finished film, and is guided by the script. Audit team oversight and quality control of the process is essential.

Arizona will be glad to provide additional information about using videos to present audit findings. Ms. Hays’ handouts are available by emailing her at: shays@auditorgen.state.az.us.

Jane Fletcher, Staff Director Education, OPPAGA, Florida

Ms. Fletcher described the Best Financial Management Practice Review of School Districts, for which OPPAGA received the NLPES Excellence in Research Methods Award. Florida has a large, growing school system, and the public has not been satisfied with school performance and typically does not favor raising school taxes. The Best Practices review was created to improve school districts’ management and use of resources. The review primarily covers support functions such as performance accountability, maintenance, and student transportation.

OPPAGA began by identifying best practices in 10 areas. The review itself begins with the school district voting to request a review and applying to OPPAGA to participate. The district conducts a self assessment, using resources available on OPPAGA’s web site, and then OPPAGA or a contractor conduct a review.

Key features of the research design and methodology include:

• Consistent statewide standards
• Flexibility to adapt the approach to the individual district
• A mixture of quantitative and qualitative research approaches
• Collection of data from the district and five peer districts
• A two-phase assessment: self assessment followed by external review

So far, four districts have been reviewed, and they were using overall 62% of the best practices. Potential savings of $65 million over 5 years have been identified.

OPPAGA encourages other states to use the information they have developed, and have placed the best practices, indicators for the practices, review guide, self assessment guide, and their own reports on their web site at http://www.oppaga.state.fl.us.

Writing for the Future

Moderator: Joel Alter, Minnesota

Speaker: Leita Hart, Leita Hart CPA & Associates

Summary:

Ms. Hart’s presentation addressed the principles for good writing and tips for creating an effective web design. Good writing principles are the same whether writing on paper or for the web. The basic criteria include:

1. Focusing on the readers by understanding what they want and need. Making your writing clear and enjoyable to read.
2. Being direct by putting the most important information first. This saves the reader time and the main point is not lost in the detail.
3. Making the information "chunky" by bringing up an issue, resolving it, and keeping the discussion to as few sentences as possible.
4. Using graphics, bullets, bolding, etc. Also, write in the active voice.
5. Using helpful titles and subtitles which are descriptive and always include a verb.

1. Smaller is always better w/ graphics and text (not too big where scrolling is necessary).
2. Strive for simplicity.
3. Build visual contrast into each page.
4. Provide a meaningful hierarchy of content.
5. Choose a limited color palette.
6. Make navigation intuitive, obvious, and fast.

Emerging Issues in Performance Measures in the States

Moderator: Kathie Schwerdtfeger, Texas State Auditor’s Office

Presenter: Wilson Campbell, GASB, Connecticut

Mr. Campbell’s presentation was about the Government Accounting Standard’s Board’s five-phased project to enhance its service efforts and accomplishments (SEA) research, also known as performance measures.

GASB History: GASB has published 2 concept statements on the topic and 10-12 papers. Mr. Campbell discussed Concept Statement 2, from 1994 titled "Service Efforts and Accomplishments Reporting." It includes a discussion on the nature of government accountability, elements of reporting, objectives, characteristics, limitations, and ways to the enhance the usefulness of information. GASB, in this statement, concluded that SEA information is an integral part of general purpose external financial reporting. However, GASB does not require it as a standard.

Performance Measures Funding and Research: The Alfred P. Sloan Foundation and the Pew Charitable Trusts have funded performance measures research projects.

There have been some published reports that compare cities and/or states on various performance measures. For example, the magazine Governing has published a special issues titled "Grading the States: A 50-State Report Card on Performance Measurement." It has also published a special issue titled "Grading the Cities: A Report Card on Urban Management."

Three major research centers for performance management include:

1. International City/County Management Association
2. Institute of Government
3. The Fund for the City of New York

GASB’s SEA Project: GASB’s research project is funded by the Sloan Foundation. GASB is 2 ½ years into the project. The five phases are:

• Establish a performance measures clearinghouse on the Internet
• Analyze and evaluate the effects of using performance measures in the decision-making process for strategic planning and budgeting, management, and reporting.
• Analyze and evaluate users’ response to performance measures.
• Develop a set of methods for communicating performance measures and distribute a guide on reporting and effective communication.
• Assess whether performance measures possess the characteristics needed for general purpose external financial reporting.

For Phase II, GASB has completed 12 case studies of how some local and state governments use performance measures. These case studies can be found on the Web site and are also available on CD-Rom. To complete these case studies, GASB’s research team conducted site visits around the country to assess the use and effects of using performance measures. Their work included interviews of administrators, programs directors, and staff; legislators and elected officials; citizens; and media representatives regarding performance measures. The research team assessed five components:

1. Background
2. Expectations for Performance Measurement
3. Actual uses and Effects of Using Performance Measurement for Decision-Making
4. Characteristics and Quality of Performance Measures
5. Expectation for the Future, Lessons Learned, Things You Would Tell Another Government about Developing a Performance Measurement System

Finally Mr. Campbell wrapped up by stating that GASB is starting Phase III - assessing users’ responses to performance measures. This phase involves interviewing elected officials, citizens, and media representatives nationwide.

Effective Contracting for Services

Moderator:  Joey Longley, Texas Sunset Advisory Commission

Ken Levine, Texas Sunset Advisory Commission

Handouts:

• "Tips on Contracting for Evaluation Services" prepared by Levine
• "RFP--A Mission Not Impossible" by Janet E. Jones, appearing in the Journal of the American Society of Legislative Clerks and Secretaries, Summer 2000

• Beware the pitfall of including too much detail in the RFP. Levine noted that in one case they included in the RFP a specific list of things to examine. He felt that the list increased their costs and reduced their future ability to be flexible.
• Levine doesn't recommend holding a conference for the bidders. He found the vendors wouldn't ask questions out of concern that they would tip their hands to their competitors. If you have to do a conference, make it mandatory for all vendors.
• Levine described the interview as an important step in his office's effort to select among the closely scored bids. He recommends giving the vendors 15 to 20 minutes for their presentations.
• He had several suggestions about the contract, such as holding back more than 10% of the total for delivery of the final product, and making intermediate payments after receiving the deliverables. He noted that most contractors had difficulty writing for the legislative audience.
• It's very important to have a staff person "hands on" manage the contract.
• Levine suggested thinking about contracting separately for pieces of the project rather than the "whole enchilada". He thinks it would have taken more work for his office, but also would have yielded a better product.
• He expects that his legislature will want to contract for more audits in the future.

Handouts:

• "Effective Contraction for Services: 'Lessons Learned'" prepared by Krell
• "Example of Request for Proposal Document" distributed by Krell

Krell said JLARC found it extremely helpful to post the RFP on their website along with key background documents and materials. Staff served as the contract managers and retained control over the overall project quality. He noted the "fine line" between recognizing the independence of the contractor and the need for a quality final product. In his office, staff scope the study and do pre-audit work to develop a detailed work statement.

Krell recommended requiring bidders to describe the approaches, method, types of analyses, etc. they propose to use. Krell said that his office includes the vendor's proposal so that it becomes part of the contract. He noted concerns about inexperienced staff, and suggested that bidders identify the major workers along with the number of hours they are scheduled to work. He also suggested asking bidders to submit excerpts from past reports, such as key recommendations and supporting rationale.

Krell recommended stating in the RFP and contract that the work needs to be of professional quality; that findings and conclusions have to be documented; that recommendations have to tie to findings and conclusions; and that the contractor has to make changes based on the technical review of organizations and agencies. I believe the contractor also has to present the report to the legislature. Still, the legislature sees the organization as accountable (particularly if questions arise later) and the project management requires a lot of staff resources "to the point where it may seem that it would save time to do the study yourself". JLARC now encourages the legislature not to require that studies be contracted.

Chuck Hefren, Florida Legislature Office of Program Policy Analysis and Government Accountability

Handout:

• "The Ins and Outs of Contracting" prepared by Hefren

Hefren said that a clear RFP is critical to achieving a quality product. The RFP describes the process and product, and established a common vision of the nature of the relationship. He suggested that the evaluation and award process should include careful development of evaluation criteria and training proposal reviewers; he noted that offices should quickly notify bidders of the selection and be prepared for public review of the evaluation score sheets.

Hefren also recommended close monitoring of the contract. Review each deliverable carefully, and make sure that payments are contingent on an acceptable deliverable. He concluded by recommending that the agency tailor its RFP, monitoring and product expectations to the specific project needs.

Discussion

Questions and discussion included how the RFP should address ownership of the working papers and final product; whether the RFP should include a non-compete provision; and policies for allowing contractors to use state rates.

Auditing Agencies That Don’t Work: Controlling the Costs of Conflict

Moderator: Jane Thesing, South Carolina

Presenter: Ralph Hasson, Chorda Conflict Management, Inc., Austin, TX

Summary:

Jane introduced the topic by presenting some situations in which audits had been requested because of organizational conflict. When the conflict within an organization becomes so pervasive that it spills into the public arena, auditors may be requested to review the situation. However, auditors tend to shy away from measuring conflict and its costs, perhaps partly because they feel uncertain of the appropriate methodology and recommendations.

Ralph’s organization, Chorda Conflict Management, specializes in the design of conflict management systems for organizations, and provides training and direct services in collaborative methods of conflict resolution. He began his presentation by summarizing some of the work that has been done to study organizational conflict. In the early 1980's, work began in negotiation skills ("getting to yes"), and mediation. Some organizations have implemented internal ombudsmen to address problems of conflict in a confidential way.

Most organizations have weak systems for managing conflict and its costs. Prevention activities are inadequate. This results in lost productivity, turnover, bad press, and high litigation costs. Ralph presented four ways to deal with conflict. These are:

1. Avoidance- no action to resolve conflict.
2. Collaboration- individual initiative, negotiation, mediation by third party.
3. Higher Authority- chain of command, arbitration, litigation.
4. Unilateral Power Play- physical violence, behind-the-scenes maneuvering.

Ralph suggested that organizations build internal systems for collaborative conflict management by stressing seven checkpoints:

Policy- organizational conflict management policy must be related to the agency’s mission.

Roles and Responsibilities - employees must be clear about their roles.

Documentation - agency policy and process must be documented and disseminated.

Selection - appropriate employees should be selected for key roles.

Training - both orientation and skills training must be provided.

Support - the organization must support its policies for conflict resolution.

Evaluation - review the process and the outcomes.

Ralph Hasson may be contacted at 512-482-0356 or by e-mail at chorda@chorda.com