NLPES Question of the Month

June-August 2006:

HAS YOUR OFFICE CONTRACTED OUT ANY OF ITS PERFORMANCE AUDIT WORK TO PRIVATE FIRMS? (Why or why not? If you have contracted, what types of work have you contracted for? How have you identified potential bidders for RFPs? How have you monitored the work and workpapers of the contractors?)

Rick Riggs, Kansas

In Kansas, we regularly contract for annual performance audits of our State Treasurer and Pooled Money Investment Board, and for certain other performance audits that are either routine or very specialized (e.g., security issues at the state lottery). We also contract for all our financial audit work, so the RFPs, contract-monitoring process, and review of workpapers are all handled the same way by the same staff.

Sometimes state law requires contracting out the work. Other times we think we don't have the expertise or available staff-hours needed.

In identifying potential bidders for performance audit work, we use our financial audit "bidders list", publish notice in the State Register (official publication of the State), and for some specialized cases ask the Division of Purchases or the auditee to give suggested potential bidders.

We require monthly status reports from the contractors and review the draft report and workpapers before accepting the final product.

Our experience, in general, is that the contractors' performance audit reports aren't as readable and clear as the ones we produce in-house, with few graphics, examples, and other tricks that we use to make our reports eye-catching and inviting for readers.

Gary VanLandingham, Florida

We have contracted some projects (mostly school district reviews) to private firms with highly mixed results. Overall, the contracted studies are much more expensive and you often don't get what you pay for.

The per-hour cost charged by private firms has generally been two to three times higher than our internal costs, and it has been very difficult to get folks to bid for projects that carry a budget of under $125,000 - consulting firms often just aren't interested in smaller jobs due to the need to cover their overhead. We have also found that we need to spend a LOT of time monitoring their work or we won't be very happy with what they produce; folks often feel it would be easier to just do the study ourselves. We developed a bid list for our school district reviews based on state data and our contacts with major firms.

Ken Levine, Texas Sunset Advisory Commission

We do not contract out, except for some occasional assistance on a project. That contract would be small and not subject to RFP requirements. We have enough staff to handle the work assigned by statute, about 25 to 30 agencies per biennium.

Keenan Konopaski, Washington

Our office has a fair amount of experience in contracting for assistance with audit work, having let contracts of various sizes related to twelve separate studies since July 2005. Most often this involves contracting out discrete pieces of a larger study. The reasons for doing so range from needing additional resources to requiring specialized expertise. Funding for such assistance is often – but not always – provided for in budgetary provisos that require the various studies. The nature of the work contracted for varies widely, with examples including:

• Gathering and synthesizing information on the structure of public health systems in other states, to provide contextual information for how our system is structured;
• Identifying best practices and alternative funding formulas related to K-12 pupil transportation funding;
• Designing and implementing a telephone survey of program clients; and
• Technical assistance in designing and reviewing a life-cycle cost model related to jail conversion costs.

Less often, our office will contract out an entire (or nearly entire) study. We recently did this on a major and complex transportation project that required significant expertise in the area of archaeological assessment.

Nearly all contracts are competitively procured. Our office maintains a listing of individuals and firms that have asked to be notified when RFPs are issued. In some cases we may identify individuals or firms at locally or at the national level that are recognized as having expertise in a particular area, and invite them to submit a proposal. And in some cases, we will advertise an RFP in the major newspapers and business journals in our state. JLARC also utilizes the services of nationwide on-line bid notification systems, such as “ework markets,” “Onvia Demandstar,” and “Bidnet”. These are firms that deliver bid notices, request for proposals and contracting opportunities customized to specific industry categories and states. Specifically, they assist government agencies create a larger and more diverse pool of suppliers. (And they are free of charge to government agencies.) Additionally, we use our state Department of General Administration’s (GA) procurement web system, and we also post notices on our own website.

We have found contract monitoring to be extremely important. To assist in this, we typically require bi-weekly, written status reports, and periodic “deliverables” throughout the contract, supplemented as necessary by in-person or telephone contact.

In general, we have found it to be more useful to hire specialized technical experts in topical areas, and we work closely with them to ensure their work is done consistent with audit standards. We have not typically contracted with companies that might be called “audit firms.”

The separately elected State Auditor in Washington State, and executive branch audit organization, has recently been authorized to conduct performance audits as the result of a citizen’s initiative. The State Auditor is intending to contract out the majority of performance audit work it is pursuing. Additional information on their program can be found at http://www.sao.wa.gov/PerformanceAudit/default.asp

John Patterson, Minnesota

In 2004, the Minnesota Office of the Legislative Auditor (OLA) contracted with two consulting firms to help us with an evaluation of Minnesota's Conservation Improvement Program (CIP). Under this program, Minnesota's electric and natural gas utilities are required to put aside a portion of their revenues for conservation activities. The utilities use these funds to subsidize the purchase of energy efficient equipment that their customers buy.

This was an extremely technical evaluation. One of the two firms provided the project team with engineering expertise. Our office has no engineers on staff. The second consulting firm specializes in energy economics. While our office has several staff with strong backgrounds in economics, the economics of utility operations and energy markets are very specialized.

OLA staff designed the evaluation, prepared the work plan, did most of the research itself, and wrote the final report. The consulting firms provided our staff with technical expertise and carried out the most technical parts of the evaluation.

We did an national search to find the consultants. One of the firms operates out of Boston, and the other operates out of Washington D.C. Our search process involved three steps. First, we listed the request for proposals (RFP) in Minnesota's State Register. Second, we identified the national association that represents these types of consulting firms, and this association send the RFP announcement to all it members. Finally, we spoke with people in Minnesota and nationally who work with these types of consulting firms and asked them what consulting firms would be interested in the type of work we were doing. We sent the RFP announcement to all the firms that these people identified. In the end, we received 14 responses to the RFP, and we had an excellent pool from which to choose the two consultants.

To monitor the work of the consultants, we took a four pronged approach. First, we spelled out in written documents very precisely the work that we wanted the two consulting firms to carry out. Second, the firms provided us with periodic updates and draft analyses that we reviewed and commented on. Third, once the firms completed a full draft report, we sent this document to all the electric and gas utilities in Minnesota and asked them to comment on the consultants' analyses. Fourth, the consultants provided us with all their working papers, which we reviewed and incorporated into our working papers.

Overall, the process worked quite well. The consultants added a level of rigor and expertise that our evaluation report would have otherwise lacked. However, going through the RFP process, selecting the consultants, and managing and the reviewing consultants' work took a lot of time.

Beth Ashcroft, Maine

Yes, we have contracted out performance audit work to private firms, or tried to, in two instances so far. The first instance was related to a review of State-wide Planning and Management of Information Technology where the contractor essentially performed the portion of the review that was a high-level risk assessment based on COBIT. In the second instance, we selected a contractor to assist with a type of cost allocation review but ended up terminating the contract because the individual the contracting firm had assigned to the project was not meeting our expectations.

We have had the best luck in identifying firms to send RFPs to by searching the Internet, drawing from our own past experiences with contractors we've used and contacting others in State government for their suggestions on contractors we might want to include in the bid list. We tried doing just a bid advertisement (ran for two weekends) on the cost allocation review and came up with no responses. We have better response rate sending notifications to specific companies.

In both instances, we designed the contracting arrangement so that as much as possible the contractor was to be seen as an extension of our department rather than being left to go in and do a review and then hand us the results. Consequently, the arrangements have involved OPEGA and the contractor acting together as a team in some respects. In this way, we have been able to monitor the contractor's work but have not resorted to needing to review all the contractor's work papers. The firm that we used to do the IT risk assessment did provide us all of their workpapers at the end of the review which is their normal course of business.

Diana Friou, Maine

Maine’s OPEGA has contracted out work. Our process has included the standard RFP process with:

• notice posed on our website and also directly sent to identified firms;
• prebidders conference for Q & A; and
• in-person interviews/presentations by top candidates.

As a new office (just completed our first year of operation) I can report specifically on one instance, our state-wide IT audit (see http://www.maine.gov/legis/opega/reports.htm). The final report describes the working relationship with the contractor. We used a contractor to perform the IT system risk assessment using the COBIT model. We combined their results in our own best-practices study. We worked as a team and so we were producing workpapers together. We characterize their final risk assessment matrix as a work paper within our higher level report.

Lynn Coulam, Texas State Auditor’s Office

The Texas State Auditor's Office statute allows for state agencies to contract certain audits. Partially paraphrased, the statute reads as follows:

“COORDINATION OF CERTAIN AUDITS.

(a) Notwithstanding any other law, a state agency, or a corporation that is dedicated to the benefit of a state agency . . . may employ a private auditor to audit the state agency or corporation only if:

(1) the agency or corporation is authorized to contract with a private entity through a delegation of authority from the state auditor; (2) the scope of the proposed audit has been submitted to the state auditor for review and comment; and (3) the services of the private auditor are procured through a competitive selection process in a manner allowed by law.

(b) At the joint direction of the lieutenant governor and the speaker of the house of representatives, the state auditor shall provide contract management services to the agency or corporation for an audit described by this section.”

Note: We delegate audit authority subject to, but not limited to the following, depending on the agency and the work being delegated: advance notice of audit related meetings, a copy of the contract and any amendments, all records we may request including copies of draft reports, and a copy of the final report.

Note: We do contract out the Statewide Financial Audit of Federal Funds.

Perry Simpson, South Carolina

We have not contracted any audit work, though we can. In our law requiring us to audit the lottery, there is a provision for us to oversee the security audit of the lottery.

John Sylvia, West Virginia

We have never contracted with another firm to conduct a performance audit for our division. There has not been a need for us to do that. We generally are able to conduct the performance audit ourselves. The state has had private companies do performance audits on certain agencies from time to time, or a state agency may contract for an independent performance audit. But with respect to performance audits that our division is responsible for, we conduct the audit.

Kerry Fitzgerald, Louisiana

Louisiana has not contracted out any of our performance audits.

Gerald Schwandt, Michigan

No, we do not contract out to private firms any of our performance audit work.