• Mission & Governance
• Services Overview
• How to Get Involved
• Standing Committees
• NCSL Foundation

• Issue Areas: A-Z
• State-Federal Relations
• NCSL Standing Committees
• News from D.C. and the States

• About State Legislatures
• Elections, Campaigns & Redistricting
• NCSL's StateConnect Directory
• Web Sites

• Online Guide to NCSL Services
• Staff Sections and Networks
• Meetings & Professional Development
• Staff Directories

• Meetings Calendar
• Online Registration
• Meetings Home Page
• Legislative Summit

• Bookstore
• State Legislatures Magazine
• NCSL's StateConnect Directory

NLPES Site Map

NLPES News

June 1999, No. 73

Inside the News

Chair's Corner
Office Profile: Michigan Office of the Auditor General
NCSL Annual Meeting: Indianapolis Invites You
Program Evaluation International

        Go Abroad, Young Evaluator!
        Motivating Democracy in Madagascar
        Special: G'day from Down-under!

Performance Evaluations -- NLPES Fall Training Conference
A View From the States:  Performance Based Budgeting --
        Louisiana Shifts Into High Gear
News From the Net

       Cybercruiting with NLPES
       Listserve Update

Small Shop Talk: Hiring Perfect Staff
NEW: Ask Eva, Lua & Tor
      (executive branch auditee tips to auditors)
News Makers
From the Editor
Calendar

Chair's Corner

Gary VanLandingham (FL)

The NLPES Executive Committee has been very busy over the past few months.  While many of the Committee's activities are discussed in other parts of this newsletter, I want to call your attention to several things:

NLPES Awards.  The NLPES awards have been modified in response to changes in our field and concerns that states have raised over the years.  Most notably, the Excellence in Evaluation award has been significantly changed, and a new award for Outstanding Contributions to Legislative Program Evaluation has been developed.   These awards, plus the traditional "Excellence in Research Methods" and "Public Impact" awards will be given at the NCSL Annual Meeting in Indianapolis.

Future NLPES Conference Sites.   The following states have been approved by the executive committee to host future fall training conferences

1999 - West Virginia;
2000 - Texas;
2001 - Kansas City (jointly hosted by Kansas and Missouri);
2002 - Arizona;
2003 - Michigan;
2004 - Florida.

Thanks to these states for volunteering!  As factors beyond our control can affect a state's ability to host the NLPES fall training conference, this list may be subject to future change.  The NLPES Executive Committee will reconfirm these states two years before their planned conference dates.  It is possible that some conferences may be co-hosted by other NCSL staff sections.  In recent years, NLPES has co-hosted training conferences with both the NCSL Research and Committee Staff section and Legal Services Staff section.

NCSL International Program.   NCSL has asked NLPES for help in signing up folks who are interested in participating in technical assistance trips to developing nations.  People who have participated in NCSL's international programs in the past indicate that the trips are hard work but are personally rewarding in terms of supporting emerging democracies.  I encourage NLPES members who are interested in this effort and who have support from their offices to submit applications.  For more information about this, see the NCSL web site.

NLPES Journal Published.  The NLPES edition of New Directions for Evaluation, a journal of the American Evaluation Association, has been published!  This edition is titled "Legislative Program Evaluation: Utilization-Driven Research for Decision Makers" and was edited by Kirk Jonas of Virginia's JLARC.  I highly recommend it to all NLPES member offices as both a training resource and a testimony to the high quality of evaluation work being done by legislative program evaluation offices.  Please join the executive committee in thanking Kirk for his work in bringing this edition to fruition.

Future Training Events.  Make plans to attend the NCSL Annual Meeting in Indianapolis where NLPES will hold several training sessions, as well as the NLPES fall conference in West Virginia. Information on the Annual Meeting is available on the NCSL web site, while a wealth of information on the fall NLPES conference is now available on West Virginia's web site.   See you there!

Back to the top

Office Profile

Michigan Office of the Auditor General

Office History:  The Michigan Office of the Auditor General (OAG) dates to 1836, the year before Michigan became the twenty-sixth state of the Union.  Since 1965, the State Constitution has provided that the Auditor General, who must be a certified public accountant, be appointed by the Legislature for an eight-year term.  Auditor General Thomas McTavish, first appointed in 1989, is serving his second term.  The Constitution also provides that the OAG conduct postfinancial and performance audits of State government operations.   Audit reports and other information about the OAG is available on our web site.

Number and Background of Staff:  Of the 152 professional audit staff members at the end of the fiscal year, 76 were certified public accountants, eight were certified internal auditors, 10 were certified information systems auditors, and two were certified management accountants.  Thirteen staff members have master's degrees, one has earned a doctorate, one has earned a Juris Doctor degree, and one is working on a doctorate.

Little Known Fact About the Auditor General: While they both served in the United States Navy, Auditor General McTavish played on the Navy Supply Corps School basketball team in Athens, Georgia, with Hall of Fame Cowboys quarterback Roger Staubach.

Recent Audit Reports:  In addition to its financial audits, the OAG completed 55 performance audits during the past fiscal year, including Oil and Natural Gas Production Reporting; Air Quality Division, Department of Environmental Quality; Children's Protective Services Program, Family Independence Agency; and Reporting of Driver License Points and the Collection and Disposition of Fines and Fees.

Dream Assignment:  To be selected as a team member for a January or February peer review of the Hawaii Legislative Auditor's office.

Difficult Assignment:  Any winter assignment in Michigan's Upper Peninsula, if you aren't a ski or snowmobile enthusiast!

Things About Michigan:  Michigan has 3,288 miles of shoreline on four of the five Great Lakes and a population of approximately 9.7 million people.  The state bird is the robin; the state flower is the apple blossom; the state tree is the white pine; and the state soil is the Kalkaska soil series.   Michigan leads the nation in the production of red tart cherries, dry beans, blueberries, pickling cucumbers, and potted geraniums.

Warm-Hearted Auditors:  OAG staff continually help others less fortunate in the community-from preparing tax returns in the VITA program to adopting entire families during the Christmas holiday season.

Oldest Office Asset:  Those same neatly wrapped but badly faded packages placed under the Christmas tree in our lobby every year!

Leisure Time Activities:  OAG staff participate in a weekly golf league, an annual golf outing, softball teams, and a Holiday Open House (complete with pot-luck food, Santa Claus, and gifts for the children).

Biggest Mistake in 1999:  Offering a suggestion to the NLPES-NEWS editor on the 'Office Profile' feature...he asked us to write the next one!

 Back to the top

NCSL Annual Meeting:
Indianapolis Invites You

Jane Thesing (SC)

Click here to view a picture of the downtown Indianapolis skyline at dusk.

Members who attend the annual NCSL meeting in Indianapolis July 24-28 will find a full agenda of NLPES events.   We plan to offer three training sessions on topics of interest to evaluators and legislators.  These include:

Use of Performance Information in the Budgeting/Oversight Process.   Managing for results is no longer just a fad.  There is growing commitment to this reform effort in both the executive and legislative branches of state governments.   This session will bring us information on the status of performance budgeting in a variety of states and tips about what is needed for success-legislative champions, good performance information, and patience!

New and Improved Evaluation Products.  Program evaluators and performance auditors are meeting the challenge of dealing with legislators' changing needs for information.  Sometimes they need quick turnaround on evaluative information and cannot wait for a full-scope review.  A quick response audit can fill the bill.  Sometimes legislators ask questions that call for consultant-like analysis.  Sometimes a positive product, such as a best practices review, is requested to help identify meaningful goals and benchmarks.  This session will provide a tour of new options for auditors who are trying to be responsive without sacrificing the quality and independence that are our trademarks.

Reviewing the Effects of Welfare Reform. States have been allowed considerable variety in their implementation of welfare reform under the federal Temporary Assistance For Needy Families (TANF) block grant program.  It's now time to evaluate the results, and our colleagues have been busy at the task.  This session will offer information on evaluation methodologies and results.  The ultimate evaluation task is a challenging one, as we attempt to determine the effects of state programs on recipients and former recipients, their children, and the community at large.

In addition to formal training sessions, NLPES will offer opportunities for socializing at an opening reception and at our annual luncheon meeting, where this year's NLPES awards will be presented.  This year the luncheon meeting will also be accompanied by a roundtable discussion of experiences in contracting out for performance audits.  There is plenty for everyone.  All NLPES members are encouraged to head to Indianapolis for this year's annual NCSL meeting.

Back to the top

Program Evaluator International

Go Abroad, Young Evaluator!

Gary VanLandingham (FL)

NCSL is likely to obtain an open-ended contract with the federal government that will result in expansion of NCSL's International Programs.  If you are not familiar with this effort, NCSL has for many years sent legislative staff to developing nations to help these countries develop democratic institutions.  In the past, the bulk of this assistance dealt with legislative operations-bill drafting, parliamentary procedures, and the like.  Consequently, NLPES members typically have not been involved with this effort.  However, this may be changing.  NCSL staff indicate that some developing countries have expressed interest in developing oversight capabilities (see Nancy Van Maren's article below).  Thus, there may be opportunities over the next few years for NLPES members to join teams of other legislative staff in international technical assistance efforts.

NCSL is asking that NLPES members who have an interest in this area to complete a "biographical data sheet" and submit it to the NCSL Washington office.  This data sheet and more information about the International Programs are available on the NCSL web site or by calling NCSL's Washington office at (202) 624-5400.

People who have participated in NCSL's International Programs in the past indicate that the trips are hard work but are very personally rewarding in terms of supporting emerging democracies.  The trips usually last 7-10 days, and folks generally get two to six months advance notice that they have been selected to be on a team.  In addition to a passport, staff obviously need authorization from their office to participate in the effort and to be away from work during the trip period.

I encourage NLPES members who are interested in this effort and who have support from their offices to submit applications.

 Back to the top

Program Evaluator International

Motivating Democracy in Madagascar

Nancy Van Maren (ID)

Click here to view a picture of downtown Tana from the hotel window.

African and Asian influences converge in Madagascar, an island country in the Indian Ocean off the coast of Southeastern Africa.  Some 14 million people from 18 different ethnic groups inhabit this island-the fourth largest in the world-of high central plains and coastal lowlands.  A French colony until 1960, Madagascar operated under a socialist dictatorship until the early 1990s when a new constitution was adopted.   Centrally-located Antananarivo, the national capital of 700,000 inhabitants (called Tana by the locals), is a mix of past abundance and socialist decay.  The remnants of remarkable architecture reside in hillside homes built long ago and framed by purple jacaranda trees, colorful bougainvillea, and intermittent rice paddies.  In Tana, people on foot, in rickshaws or in carts pulled by Malagasy oxen (called zébu), compete in the haze of smoky auto exhaust with an endless supply of Deux Chevaux taxis flooding the streets on their way to and from markets.

I traveled to Madagascar last December to take part in NCSL's three-month Malagasy National Assembly Capacity Enhancement Project (CEP).  The project, funded by the U.S. Agency for International Development, brought American state legislators and legislative staff to Tana to work with members of the Malagasy National Assembly and their staff.  The goal was to strengthen the functioning of Madagascar's national legislative branch of government, particularly in light of the country's strong executive, the former military dictator-but-now-popularly-elected, President Ratsiraka.   Adding urgency to the cause was a constitutional amendment added some months earlier that gave the president the power to dissolve the National Assembly and reduce the already limited authority of the legislative branch.

Using a series of teams, the CEP provided a new member orientation, a diagnostic study of the Assembly's committee system, technical assistance on libraries and research services, a seminar on legislative ethics, help in organizing open legislative meetings, a study tour to South Africa and Bénin, instruction on legislative-press relations, and a workshop on legislative committees.  Our three-member team, which included Rep. John Martin from Maine and Hannah Shostack, a Senior Research Associate from New Jersey, conducted the workshop on legislative committees.  My role was to convey the concepts of independent, nonpartisan oversight, and "checks and balances." Governing with checks and balances, so familiar in the U.S., was foreign to Malagasy Assembly members who are unused to challenging executive privileges.  During the last Assembly (between 1992 and 1998), no more than six of the 60 bills enacted were legislative (rather than executive) initiatives.

Results?  The concepts of oversight and accountability took root for workshop participants through a series of thoughtful questions on how these mechanisms work within their governmental context, and during a lively role play in which a legislative committee grilled government, business and public representatives about a bill.  For the first time, the National Assembly held two open committee meetings with the CEP's organizational expertise and encouragement, despite the current Assembly's rules that close legislative committee proceedings to the public.  Also, although serious deliberation of government bills is virtually unknown, with the CEP's assistance, members of the National Assembly prepared a plan for providing needed resources for encouraging productive committee deliberations.  From the new member orientation to the roundtable on press relations, Assembly members and staff directors also developed plans for recruiting and developing a qualified, nonpartisan staff--personnel presently nonexistent in Madagascar.

Click here to view a group picture from NCSL's Malagasy National Assembly Capacity Enhancement Project.

Despite these hopeful results, it is clear that the Malagasy National Assembly faces the ongoing challenge of leveraging its members' skills and the technical assistance we provided to ensure long-term progress.   In a country where media coverage is guaranteed only when the media are transported to the event with expenses paid, it is doubtful that active public participation with the elected Assembly is around the corner.  Furthermore, the Assembly will have to work at retaining forward-thinking and ambitious members in an environment where executive positions are sought after advancements due to their greater influence.

Although it is difficult to visualize the long-term results of these efforts, exposure to good government standards and methods, combined with NCSL's growing list of legislative contacts, plant the seeds of opportunity for a more thoroughly democratic process in the future.  Sincere thanks are due NCSL for its efforts to link the skills we take for granted with existing needs internationally.  I am personally grateful to have been part of this ambitious project (and to have been given the opportunity to see and hear the Indri lemurs in their only indigenous home in the world).  

Jeremy Meadows, Senior Policy Specialist, NCSL, graciously contributed to this article.

Back to the top

Program Evaluator International

Special: G'day from Down-under!

Victorian Auditor-General's Office, Australia

Editors Note: In this first article on performance evaluation abroad, I was awe-struck by the popularity of this function in Australia! The torrent of public support described for this shop when its mission was being re-evaluated is absolutely unheard of in the U.S.-to my knowledge.  If I'm wrong, please send your state shop's bumper sticker to me.  I want to start a collection!  Thanks to Barbara Mountjouris and staff for the article, and keep an eye out for Barbara who may attend the 1999 NCSL Conference in Indianapolis.

Victoria is one of the seven States and Territories that make up the Commonwealth of Australia.  (Just in case you didn't know, Australia is roughly the size of the U.S., excluding Alaska).  We're a small state located on the southeast corner of the continent-only 3 per cent of the total size of Australia-but we have about 25 percent of the country's population, about 5 million Victorians.  (We're roughly the same size as Minnesota.)

Click here to view a picture of Melbourne-our hometown.

Victoria's capital is Melbourne, which is where our office is based.  Melbourne has a population of about 3.5 million and is often referred to as the "cultural capital" of Australia.  Our National Gallery is the home of Jackson Pollack's "Blue Poles", and the variety of our top class restaurants reflects our diverse multicultural society.

We're also keen on our sport, and Melbourne is blessed with a multitude of world-class sporting venues.  Our Melbourne Cricket Ground packs in 100,000 plus at a good game of Aussie Rules football in the winter, or a thrilling cricket match against our traditional foe, England, in the summer.  Summer also sees the Melbourne Tennis Centre host the Australian Open, and in March each year, our Moomba Festival and the Formula 1 Grand Prix really get the town buzzing.  Add to these our Spring Racing Carnival, culminating in the Melbourne Cup in November, and we are occupied throughout the year, though we still find some time to do a bit of auditing-to add that extra sparkle to our lives.

The Victorian Auditor-General's Office was established in 1851.  It has a long and unrivaled tradition as an independent watchdog servicing the needs of the Victorian Parliament and, through the parliament, the people of Victoria.   Our corporate motto is "Auditing in the Public Interest".

Running Victoria is a big business.  The public sector has an annual expenditure of about $47.2 billion and an asset base of around $85.2 billion.   We are responsible for the financial audit of about 550 organisations, including government departments, public authorities and municipal councils.  Recent performance audit topics have included public transport reforms, Victoria's gaming industry, and marketing government services.  The results of our financial and performance audits are presented to the parliament through our Auditor-General's reports, which can number 10 to 12 in any one year.

In late 1997, the Victorian government initiated a review of our principal legislation, the Audit Act 1994.  The review sought to determine whether all audits conducted by the office should be subject to contestability.  (Prior to the review, audits were conducted using a mix of internal resources and externally contracted resources-as determined by the auditor-general.)  The review generated an intense level of debate in parliament as well as within the auditing profession, the Victorian community and, above all, the media, as it was widely perceived as a threat to the independent role of the auditor-general.  As an office, we were caught up in this process for about 15 months, and our media management and community relations skills went through a very rapid learning curve.

The review resulted in legislation being passed in December 1997 that essentially precludes the office from engaging in any direct auditing, with all financial statement and performance audits to be contracted out.  The new audit arrangements came into effect in July 1998.  The auditor-general retains responsibility for signing off on all public sector financial statement audits and for planning and reporting the results of performance audits to the parliament.  As far as we know, the new legislative arrangements are unique under the Westminster system of government worldwide.

Before the legislative changes were introduced, our office had about 150 staff, with the majority performing financial and performance audits.  Now, we have about 50 full-time staff working in financial audit, performance audit and in corporate operations.  The balance of the staff were transferred to a new government-run statutory body known as "Audit Victoria," which actively participates in the contestability process for public sector audits along with private sector providers. About 60 percent of our current staff hold qualifications in accounting, business, social sciences, and economics.

While we still perform the important quality control processes over contracted financial and performance audits and continue to prepare and present our reports to the parliament, we are in the process of evaluating our new skill requirements in light of the changed environment, for example, greater emphasis on tendering and contract management practices.

Our current auditor-general, Ches Baragwanath, was appointed in August 1988 and is the fourth longest serving Victorian Auditor-General.  He has recently announced his retirement and will step down at the end of July.

His straight-shooting style and revealing reports to the Parliament have ensured a high public profile for a man who indicated that he intended to be a "faceless bureaucrat" when first appointed.  In fact, he has become a virtual cult figure in Victoria, being all things to all people.  Some notable achievements include:

• He is almost a permanent feature in daily newspaper editorials (latest count was well over 100 editorials since assuming office-perhaps a world record!).
• He was selected as "Accountant of the Year" in 1990.
• He has become a reference point on fine dining with regular mentions in Victoria's "Good Food Guide."
• He receives water bills from little old ladies asking him to check the calculations.
• He is often called upon by conservationists to save endangered forests.
• He is the person you turn to for lost property, as was the case involving a government car that was left at the beach and allegedly "floated out to sea."
• He has been asked to personally count all attendees at Melbourne's Formula 1 Grand Prix to settle a claim of exaggerated attendance figures.

One of the most significant developments over the years has been the increasing level of community awareness and appreciation of the work of the Office.   This was evident during the recent changes to the office's audit legislation when mass meetings took place throughout Victoria to debate the issues surrounding the proposed changes.  A government member of parliament was moved to resign over the issue, and the proposed changes were put forward as one of the main reasons for the government losing a by-election in 1997.  (Many cars were seen driving around Melbourne with "Vote 1-Auditor-General" and "Protect the Auditor-General" bumper stickers at the time.)

Click here to view a picture of the Victorian community coming out in the thousands to attend public meetings during the Audit Act Review.

We even saw a candlelight vigil on the steps of parliament house in support of the auditor-general and his office, and thousands of petitions were circulated around offices and at shopping centres.  We got the feeling that the community would go to great lengths to protect its auditor-general-and that's a nice feeling to have.  In many ways, this was testimony to the hard work the office puts into ensuring that it has a positive and respected profile in the community.

Click here to view a picture of our fun-packed "audit" exhibit at the Royal Melbourne Show.

Each year we "take the office to the people" at the Royal Melbourne Show (similar to your state fairs).  Yes, we have a big exhibit at the show, which attracts many thousands of visitors, but we don't go in for educating visitors about the finer points of auditing.  Over the 10 days of the show, volunteer office staff have a chance to interact with the community and talk about our role.   It's great to hear, first-hand, how much the community appreciates the work we do on their behalf.  The show is mainly for families, and we adopt a fun atmosphere on our exhibit with thematic audit games, competitions, prizes and plenty of colour, noise and excitement.  We even feature Strawberry the Clown (just think of Krusty the Clown from the Simpsons and you'll get the idea), who is a regular on our exhibit.   Last year, Strawberry was asked, "Are you the auditor-general?" by a 9-year-old visitor!  Strawberry was left speechless and was unable to juggle for at least 2 hours.

Our office has developed a number of strategies to ensure that its interaction with the community, the media, parliamentarians and professional associations delivers the most favourable outcome for the office.  For instance, our Internet site is a prime communication medium and contains office information such as our award-winning annual report, as well as the contents of our very important reports to parliament.   Our reports are released on the site as soon as they are tabled in the parliament.   Media representatives and the general public regularly utilise the site for "tabling time" releases, as well as in-depth research at a later date.  Why not pay our web site a visit?  Our site has links to 47 US audit office web sites, and we'd love to have our link on your site.   Or e-mail us at vicaud1@vicnet.net.au.

So, as you can see, we are a long-established, traditional audit office that is undergoing a significant change in its operations, but we're still hanging in there!

We hope you enjoyed visiting our office.

Back to the top

Performance Evaluations:
Improving Government Not Just Criticizing Government

NLPES Fall Training Conference
Marriott Town Center, Charleston, West Virginia
October 6-9, 1999

It's time to start planning to attend this fall's NLPES conference, hosted by the West Virginia Office of the Legislative Auditor Performance Evaluation and Research Division.  The newly appointed Comptroller General of the United States, David Walker, will give the conference keynote address. The preliminary agenda includes the following topics:

• Policy Environment of Legislative Performance Evaluation (Dr. Daniel Fenn, Harvard Kennedy School of Government)
• The Pursuit of Absolute Integrity: How Corruption Control Makes Government Ineffective (Dr. Frank Anechiarico, Hamilton College)
• Can Everything Be Measured? Creative Measurements of Performance
• Comparing Your State's Program to the National or Regional Average: Are Averages Really Meaningful?
• Agencies' Perspective On Being Evaluated: Do Evaluators Treat Agency Heads As If They Are Darth Vader? And, Are Evaluators As Dumb As Agency Heads Think They Are?
• Legislators' Perspective of Performance Evaluations: Are Evaluations Important? How Could They Be Improved? Are Reports Too Long and Too Complicated?
• The Audit Follow-Up: Do Evaluations Change Agency Performance Without One?
• Usable Information: How to Increase the Impact of Performance Evaluation
• Poor Performance Evaluations: Causes and Cures
• The Scientific Model of Evaluation-Using The Null Hypothesis: How To Avoid Wandering Around the Desert for 40 Years

OK, if you're already looking for the registration form RIGHT NOW, consider a mention of the ultimate in West Virginia entertainment.  This is one of the state's greatest natural resources, and we're not talking about coal mining. Splash!  It's white water rafting!  When you've got a good guide, a sunny day and sparkling H2O roller coasters, you'll be set for an optional activity you won't soon forget.  There's also golfing, shopping and mountain biking for those who prefer using the word "rapid" for a reply to audit findings.

Questions?  Contact Antonio Jones.   Plan to come and enjoy yourself!

Back to the top

A View From the States

Performance Based Budgeting:
Louisiana Shifts Into High Gear

Gerald Hoppmann (LA)

Louisiana took the first step toward performance based budgeting when its legislature passed Act 814 in 1987.  Act 814 required the state to adopt a program budgeting system.  Previously, appropriations in the state were based on expenditure categories, such as salaries and operating expenses.  Act 814 was different in that it required budgeting by program, as well as the use of objectives to show an agency's program achievements.

As a result of Act 814, the state began to fund individual programs and moved away from appropriating funds based on expenditure categories.  Program budgeting was viewed by many as a way to provide additional flexibility to program managers, although there was some concern by the legislature and the public that the process lacked accountability.  Act 814 required the use of objectives to show an agency's program achievements, but no clear message was sent to agencies about how to develop and report goals, objectives, and performance indicators.  As a result, performance data did not play a significant role in the appropriations process until 1995.

It wasn't until the passage of Act 1100 in 1995 that Louisiana became serious about funding programs based on performance.  Act 1100 required agencies to develop specific program goals, objectives and measures of performance, which could then be used by the legislature to make funding decisions.  The act also established the "Louisiana Performance Audit Program" and authorized the Office of the Legislative Auditor to make annual recommendations about the effectiveness and efficiency of programs and services provided by agencies.

As a result of Act 1100, the Office of the Legislative Auditor conducted performance audits of program authority and performance data of all executive branch departments.  The audits showed that incomplete and otherwise useless performance data were reported to the legislature, which had a negative impact on the budget process.  Recommendations focused on the need for additional training for agency planning staff and better orchestrated communication between the legislature, Office of Planning and Budget, and individual agencies when developing performance data.   Based on this work, the quality of performance data used in the budgetary purposes improved.

The success of performance based budgeting in Louisiana was enhanced by the passage of additional legislation in 1997.  Act 1403 mandated agencies to include missions, goals, objectives, and performance indicators in short-range planning documents, and required the Office of Planning and Budget to report programs, objectives, and performance indicators in the executive budget.  Act 1465 required long-range strategic planning and quarterly reporting by state agencies, and it required the appropriation bill to include key objectives and performance indicators.  It also provided rewards and penalties for agencies that attain or do not attain targeted levels of performance.

Although Louisiana got off to a slow start, it has moved forward significantly with the passage of "performance oriented" legislation in 1995 and 1997.   Legislative oversight and leadership is essential for a successful implementation of performance based budgeting.  We in Louisiana are hopeful that the momentum of the last few years will carry us forward into the new millennium with continued success.

Gerald Hoppmann is the Performance Audit Manager of the Louisiana Office of the Legislative Auditor.

Back to the top

News From the Net

Cybercruiting with NLPES

Steven Birnholz (FL)

Has your office been posting its job announcements on the NLPES web site?  If not, your recruiting arsenal is lacking a very effective (and FREE) tool.

Since its creation in September, the NLPES Job Listings page has become the third most popular gateway to the site after the home page.  In fact, half of the posted position announcements have received more than 50 hits (Internet requests) apiece.

Why all the excitement?  Put simply, today's job seekers are taking their searches online.  The Internet provides candidates with a fast, inexpensive means of browsing job listings from around the country.

As a result, online recruiting, or cybercruiting, is the fastest growing trend in the employment field.  Numerous companies are using cybercruiting, either alone or in combination with traditional recruiting methods, to greatly expand the breadth of their recruiting efforts.  According to two recent human resource management studies, the practice of cybercruiting has more than tripled over the last three years with 70% of human resource departments now using the Internet for job announcements.

This explosive growth in cybercruiting can be attributed to the practice's many benefits.  Companies cite lower recruiting costs, expanded reach, decreased search time, and better candidates as the main reasons for increasing their cybercruiting efforts.

• Lower Recruiting Costs.  Costs for posting Internet job announcements are minimal, especially compared with the typically high costs of placing newspaper ads, attending job fairs, and mass-mailing announcements.  Placing your office's announcement on the NLPES web site won't cost you a dime!  (Despite the site's multi-million-dollar budget, we draw the line at paying you for the honor of your ad's presence.)
• Expanded Reach.  Tired of receiving applications from the same people seeing the same advertisement in the same local paper every time your office has an opening?   Cybercruiting places your office a mouse-click away from everyone on the planet.   The bigger your candidate pool is, the better your chances are of finding that golden hire.
• Decreased Search Time.  Given that the best job seekers generally find a position within a week of search initiation, it is vitally important that recruiters "spread the word" as quickly and broadly as possible.  Research shows that cybercruiting can cut weeks off an employer's recruiting cycle.
• Better Candidates. Studies show that Internet users tend to be better educated and more computer literate than the general population.  These qualities are often sought in candidates for policy evaluation positions.   Furthermore, cybercruiting often yields applicants who are better "matches" for positions listed since recruiting organizations can post detailed descriptive information on their web sites that candidates can evaluate before deciding to apply.

Outstanding potential employees for your office are probably surfing the web at this very moment.  Don't delay-post your position announcements on the NLPES web site as soon as possible.  The benefits of cybercruiting are clear, and the cost to your office is nothing.  Contact Steven Birnholz, or call (850) 487-3631, to broadcast your position opening to the world.

For more information about cybercruiting, navigate to the Recruiter's Network, Public Sector Jobs, and Logos Internet Intelligence.

Back to the top

Small Shop Talk:

Hiring Perfect Staff

Cynthia Johnson (NE)

What manager of a small program evaluation group in the market for a new staffer has not wanted to run an ad just like this one:

Those of us who work in small shops quietly recognize that unwritten requirement for almost superhuman ability in our evaluators.  We look enviously, at times, at offices around the country that have the luxury of hiring staff who will specialize-some doing research, others writing, developing statistical methodologies, or making presentations at public hearings.

Since we don't have the benefit of staff specialization, recruiting and hiring take on heightened importance.  And the ramifications of hiring mistakes are the stuff of nightmares.

If you have a staff of four people and one of them isn't firing (at least partially) on all cylinders, that means the effectiveness of your office is reduced by up to 25% off the top.  Then, to boot, staff morale suffers as those who are more broadly functional pick up the load dropped by the less-competent staffer.

To make matters worse, trying to assess whether a new hire has broad-based abilities can take quite awhile when projects are completed at the rate of only two or three a year.

In making those all-important hiring decisions, small shop managers have to do much more than just cross their fingers and hope.  And, unless luck is with them, a simple interview and writing test probably won't do it either.

Over time, we who work in the Nebraska Legislature's Program Evaluation Unit have developed a hiring test that has three components in addition to the in-person interview.  So far, this system has worked well to separate the good from the not-so-good during the hiring process.

The first part of the "test" is administered to the candidates at the outset, prior to the interview.  It consists of a "personality" or "working-style" inventory that we developed in-house.   This instrument asks candidates to respond to questions for which there are no "right answers."  Questions like:

• When working on a long-term project, I prefer to (select one): __ work alone; __ work with others.
• Rank the following report-preparation functions in the order that you think demonstrates where your strengths lie (1=strongest, 4=least strong): __ writing; __ organizing; __ editing/revising the work of others; __ proofreading.
• How would those who know you professionally (or academically) describe you? (select three.): __ detail oriented; __ friendly; __ concept oriented; __ flexible; __ persevering; __ having common sense; __highly organized; __assertive.

While there are, technically, no "right" answers to the foregoing questions and statements, the responses given can go a long way toward demonstrating how a candidate would fit into the evaluation staff.  Obviously, the responses that are judged "best" will change as the staff changes and talents are lost and gained.

Once the interview is completed, candidates are asked to complete an editing and a writing test -both of which require more than a little ability.

The editing test is based on a two-page, very badly written letter that requires organizational change, elimination of extraneous material, sentence crafting and simple proofreading.  Candidates are allowed to complete the test using pen and paper or on the computer.

The writing test-which was administered in Nebraska for the first time last fall and was quite successful in differentiating the abilities of those who took it-was developed by Stan Stenersen of the GAO Seattle office. (You probably know Stan as the fellow who has presented excellent writing workshops at the NCSL annual meeting and elsewhere.)  The test proposes a "mock" evaluation in which the candidates are given a description of their audience (state legislators) and relevant background information.  In addition, the candidates are given a list of data bits-information like that obtained during the research phase of an evaluation.  They are then asked to write a summary evaluation utilizing the background information and data bits.

The test requires those who take it not only to write well but to be able to categorize, recognize patterns, and draw conclusions based on the data.   Consequently, it works much better than a simple writing test to weed out those who have difficulty with some of the most important abilities needed in evaluators.

Overall, this fairly comprehensive approach to the hiring process has been successfully used with three new staffers in Nebraska.

Now, if we could just figure how to recruit candidates to a state that has no mountains, no ocean, no great lakes and pretty extreme weather.

Cynthia Johnson is director of the Nebraska Legislative Program Evaluation Unit.

Back to the top

Ask Eva, Lua & Tor

Do you have any tips for auditors on what they should do or say to increase the chances for obtaining openness and cooperation from auditees?  And, conversely, are there any "red flags" or things auditors should never do or say because they will have a negative effect on openness and cooperation?

Eva, Lua & Tor Comment:

Keep the lines of communication open. To increase cooperation, keep the lines of communication open.  If an auditor requests information, and more information, and more information but never provides feedback about how the information is being used or what conclusions are being made, feelings of discomfort increase.  This tends to lead to less cooperation-if only due to human nature.  Also, don't spend a lot of time onsite and then "disappear" for weeks or months at a time.  If it applies, try letting the auditee know things are delayed for x, y, or z reason.   Then, when you have to do follow-up, your phone call is not a shot from out of the blue.

Avoid the fear of administrative retribution.  Be aware that this environment exists and put a little thought into it.  Consider communicating to executive branch employees involved in an audit, the thought, "Feel free to tell me anything, and if it concerns issues you don't want me to attribute to you, then I won't, and I will try to find another way to confirm what you say is true."   Be aware that there can be good information this way, but review it carefully to avoid personal sour grapes or non issues. Eva: I confess that I worked on befriending auditees in order to gain their confidence.  I was often sympathetic.  I think it was real and not trickery.  People seem to open up to people like that.   Also, I tried to remain sensitive to concerns of retribution.  I never REALLY appreciated those fears until coming into the executive branch, though.

Read requested reports and analyze agency data carefully.   Auditors typically request voluminous amounts of agency materials and whether that material is useful or even used is sometimes left questionable for auditees. Lua: We have provided auditors with detailed and complicated information that has taken substantial resources to compile, only to have it ignored.  This is obvious when only a week or two later, they ask for the same information.  I would be happy if an auditor called back to acknowledge reading requested information, even if the data were not going to be used in a report.

Attitude and body language make a big difference.  It is much easier to deal with auditors who display a positive, helpful attitude than those who display a patronizing, know-it-all personality.  Avoid a "holier-than-thou" posture. Tor: Sometimes it is a way of coming across like you know better than the person you're interviewing-a sort of smugness- that gives off the impression that you've already formed conclusions.  Why should an auditee cooperate when the auditor already knows what they are going to say, or at least they act like it.

Take individual auditees seriously and really listen.  Some "red flag" issues include: an auditor's indication that they don't take an auditee seriously; that auditees' names will appear somewhere that they don't want; or the auditor is taking all the information you gather directly to agency management to verify it. Lua: Now that I am on the other side, it is obvious that many times auditors don't listen carefully unless they are being told what they want to hear.   Try to give careful consideration to evidence presented by auditees that doesn't necessarily support your premises or conclusions.

Change a word or two.  Sometimes all that is needed to come to an agreement on a draft finding is changing a word or two.  Such changes can be painless for the auditor but provide a lot of goodwill and relief for the auditee.

Back to the top

News Makers

Back to the top

From the Editor

Mark Bucherl (IN)

Contributors make my day!  And if it wasn't for pesky deadlines, I could keep collecting the latest arriving additions for months; but...the show must go on, and the newsletter must be published.   Thank you!

This issue is truly the first with major "international perspective" though this difference occurred by coincidence.  Last fall, I e-mailed several chaps in the U.K. National Audit Office, and no one replied.  Nice web site, though.  But, after a few e-mail exchanges with the Victorian Auditor-General's Office, we received some of their down-under evaluation experience.   And Nancy Van Maren's Madagascar trip highlights NCSL's newest pitch to program analysts to go abroad.

We say goodbye (at least in person) to editorial associate Kristin Breen from this office, as she will soon be moving to Virginia.  Kristin says she will be available via e-mail for editorial assistance.  I see a potential bandwidth problem here, because this newsletter is not a one-person operation.  Added critical eye thanks to Dick Sheets, Beverly Holloway, Phyllis McCormack, and my morning-sick wife Stacy (Y2K baby?) for support.

Lastly, COME to Indianapolis for the NCSL Annual Meeting!  Helping with the conference hospitality committee, I can tell you the red carpet will be wide and plush and the sessions wonderful!  OK, the Indy 500 opening social evening will probably be a happenin' event, too (just wait until Indy hosts Formula 1)!  And make sure you shop our new downtown Circle Centre mall!   See you here soon.

Back to the top

NLPES Calendar

For more information, email Bob Boerner or call NCSL at (303) 364-7700.

Back to the top