2002 NLPES-Fall Training Conference Notes
Crosscutting Issues
Moderator, Jane Thesing, Assistant Director, South Carolina Legislative Audit Council
Panelists:
- Jane Bailey, Audit Supervisor, Nevada Legislative Counsel Bureau - Audit Division
- Ned Parrish, Principal Performance Evaluator, Idaho Office of Performance Evaluations
- Gary Brown, Audit Division, Administrator, Michigan Office of the Auditor General
Thesing commented that some of the South Carolina Legislative Audit Council's best work involved multiple agency reviews, adding that this type of study frequently has the most impact.
Bailey proceeded with her presentation.
She indicated that in her state, issues are selected by the Legislature, and generally selected to address some problem facing the state. Within that issue, the Audit Division has the ability to address their audit scope.
Some of their Division's audits are 'secondary research' in that they summarize other audit report findings to reach conclusions. Specifically, they maintain a database of their Division's past findings, allowing audit staff to search to identify similar findings. She also added that many large issues are frequently addressed in other states.
She had a number of suggestions to assist any audit agency planning for an audit involving issues that cut across several state agencies. First, these are by nature big audits and therefore it does not pay to 'cut corners'. Be sure to check your data to ensure it is what you think it is. To help this, she suggests auditors send sample surveys to all agencies involved in the audit to see what data or other information they have that may be helpful. She also suggests that in the planning stages, auditors should take the time to develop complete and understandable audit plans. Further, it is a good idea to define the population involved in any statistical analysis well before analyzing the data. Also, in order to lend weight to your findings, auditors should develop financial impact estimates for any recommendations.
If the auditors develop their own database, there are a number of things they should remember. First, test the data setup to ensure it really addresses the audit question. Also, be sure to test any database with dummy data before entering real data. In addition, if multiple people are going to enter that data, be sure they are all 'on the same page' for data entry decisions. One way to do this is to develop data entry screens that give categories for input, rather than allowing staff to type the data in by hand. This forces data entry staff to use the same terms. She also suggests the teams have frequent reviews or meetings to ensure consistency.
Bailey discussed several obstacles auditors might encounter when researching crosscutting issues. First, auditors must decide how they are going to organize their audit --- she believes that it is better if the Legislature generalizes the goals of the audit as much as possible. Also, when auditors develop multiple recommendations, it helps to have a 'lead guy' in each agency to help follow up on their progress implementing the recommendations. These recommendations typically take longer to implement. She also noted that one disadvantage to audits involving issues that cut across many agencies is that they take a great deal of audit time to study. They also can involve elected officials.
However, there are two major advantages to analyzing crosscutting issues. Auditors can get better support for changes. Further, the involvement of multiple agencies means that there is a support mechanism for smaller agencies trying to implement the recommendations. Finally, large, system-wide recommendations can result in greater cost savings.
Parrish spoke next.
He indicated that the Idaho Office of Performance Evaluations is a very small office and is relatively new. He discussed several of his Office's past evaluations that involved issues spanning multiple agencies. For example, his office studied travel management, which looked at state agency policies and practices regarding travel, and tried to identify potential savings to the state. They also looked at how much state agencies charge their employees for state-owned housing.
In Idaho, their oversight committee selects audit topics, which frequently look at potential budget savings. During preliminary survey, audit staff look at the availability of crosscutting data in order to identify potential scope limitations. They may have to go back to their legislative committee to revise the scope of the audit if reliable data is unavailable.
Some issues complicate multi-agency reviews. Complex projects involving multiple state agencies complicate the communication process. The more agencies involved in the review, the more people auditors have to meet and update. In addition, auditors may have to deal with multiple databases, which mean even more time needed to evaluate and adapt agency data.
One possible way of saving time is to make use of statewide data where possible. For example, state accounting systems, state fixed assets and other statewide information may be available in place of individual agency data. However, sometimes there is no statewide data available and you have to collect the information yourself.
Therefore, he also recommends limiting the scope to a subset of the data. Where possible, focusing on larger agencies or a representative sample of the agencies might limit the data collection demands.
He has also used surveys to collect standardized data or other information from all agencies. He pointed out that if auditors do select this method, it is important to pre-test the instrument first.
Parrish also talked about the difficulties in communicating with multiple agencies during a crosscutting audit. His agency sends out an initial mass mailing to all agencies involved in the audit informing them of the audit scope. They also have conferences with the agencies at the beginning and the end of the audit, but tend to focus only on the major agencies involved during the audit.
He added that the crosscutting audits are typically large ones. For example, 6 of the 10 largest audits conducted by his office involved crosscutting issues. In addition, sometimes they run into difficulties with the inconsistent practices of some agencies. For example, his travel management audit was complicated in that not every agency had a single fleet manager. Some had fleet managers assigned at the program level.
Gary Brown then spoke.
The Michigan Office of the Auditor General has six divisions, one of which handles unusual audit requests -- including audits that cover multiple state agencies. Some types of issues they explored in the past were oil and natural gas production reporting systems, and driver's license points and fees (which reviewed the activities of multiple branches of state government).
His Office does not have a Legislative audit oversight committee. Instead, his unit takes into account the wishes of their committee chairs and makes up its own audit schedule.
One thing he does when working on crosscutting issues is add more emphasis on standardized procedures. This stems from their belief that it is important to treat all agencies the same. In addition, the Office tends to form sub-teams. In a sub-team environment, it makes sense to stress standardized procedures.
Like the other presenters, Brown noted that it was cumbersome coordinating communications with all of the agencies involved in crosscutting issues. He added that it also makes it difficult getting them to agree on solutions to the identified problems.
He believes that there is significant value in addressing crosscutting issues. First, the Office can reach stronger and more comprehensive solutions by looking at the entire system. Also, by dealing with multiple agencies, the Office can increase inter-agency cooperation. Finally, crosscutting issues can often address weakly performing programs from a holistic approach. Sometimes agencies have multiple priorities that hinder their performance. Looking at crosscutting issues can address these.
 2002
Fall Training Conference Notes
|
