NLPES Site Map
Arizona Office of the Auditor General
Performance Audit Division
Standard Audit Program Objective
for Testing Data Reliability
Objective I: To determine the reliability of computer generated
data for audit purposes. Reliability is defined as data that are accurate,
timely, and complete. (Note: Review Yellow Book standard 6.62, and GAO
publication Assessing the Reliability of Computer-Processed Data,
prior to beginning this objective.)
Issue IA: What is the relative level of risk of using computerized
data, and will additional test work be necessary?
A. Review fieldwork audit objectives and identify those that
require obtaining and using computer generated data. Include (1) data that
will be provided by the agency, (2) data to be created or compiled by Auditor
General staff, and (3) data that will be obtained from other sources, such
as counties and agency contractors.
B. For each data source identified in step A, document the audit purpose
to be served by the data. For example, will the data be reported as
general background, or will it be analyzed to evaluate the agency's performance
regarding an audit issue? Will it be reported in a separate, detailed appendix?
C. Assess the criticality to the audit of each data source.
D. In consultation with team leader, audit manager, and methodologist,
determine whether the remainder of Objective I is applicable to each data
source. Examples of data sources that may be considered for termination
at this point include: data that will not be used, based on lack of criticality
to the audit; data from the agency's accounting system, which is subject
to regular financial audits; and possibly some data from sources not accessible
to auditors, such as federal agencies or multi-state criminal record networks.
Issue IB: In conjunction with IC, for each data source identified
above, obtain descriptive information to facilitate obtaining, analyzing,
and using the data.. Interview appropriate people and obtain relevant documentation
(such as a codebook). Include the following items.
-
Type of data. Examples: Excel spreadsheet, Access database, Oracle
database.
-
Structure of data. Example: database field list and description
of relationships between databases.
-
Information required to interpret the data. Example: meaning
of field names, dictionary of codes used in fields (e.g., "D" for "discharged"
in "patient status" field).
Issue IC: For each data source identified in IA, obtain information
necessary as a basis for assessing the extent to which direct test work
should be performed.
A. Obtain general information on the control environment:
-
Use of data. How the data are used by the agency. Also, who else
(outside the agency) uses the data, and for what purpose? Obtain sample
reports as appropriate.
-
Known weaknesses of data. Have interviews or auditor review of agency
information disclosed any control weaknesses? Did users report concerns
about accuracy of data or suitability for the way the agency uses it? Have
auditors been informed of recent changes in the structure or procedures
that could suggest previous data validation studies are no longer pertinent?
B. In consultation with team leader, manager, and methodologist,
assess the need to go on to step C for this data source. For example, it
may be unnecessary to further test the control environment if step A reveals
substantial weaknesses.
C. Obtain more detailed information on controls over the data source:
-
Data entry. Who does the data entry, and what is the source of their
information (e.g., case manager enters key dates from own handwritten case
notes)? Are there procedures for checking data entry? What edit checks
exist to prevent data entry errors?
-
Access to data. This can be a narrative description or a list. In
either case, identify all those with access to the data (including people
outside the organization), and describe their level of access to the data
(e.g., can they query the data, can they enter new data, can they change
existing data).
-
Developer and programmer information (if known). Who initially developed
the data source, and who is currently responsible for making any necessary
programming changes.
Issue ID: Develop and implement a test plan for each data source.
A. In consultation with team leader, manager, methodologist,
and IT staff, develop a plan for conducting specific test work to determine
data reliability. Consider:
-
Strength of controls
-
Risk of error and omission
-
Any other issues and concerns that might affect data reliability
B. Document the basis of the test plan. (See page 19 of the GAO
publication, Assessing the Reliability of Computer-Processed Data.)
C. Develop specific audit steps for carrying out the test plan.
D. Review the test plan and audit steps with IT staff and the methodologist.
E. Perform the steps and assess the results. If necessary, revise the
test plan, and carry out any required additional steps.
Issue IE: Conclude on reliability of data, and prepare documentation
for workpapers.
A.Summarize the testwork and the results. Identify barriers
to testwork and any limitations.
B. Draw conclusions about the reliability of the data. Document conclusions.
C. Based on audit purpose of the data and data reliability, determine
whether and how to use and report the data. Document decision.
D. Review results and conclusions with team leader, manager, methodologist,
and IT staff, as appropriate.
|
