• Mission & Governance
• Services Overview
• How to Get Involved
• Standing Committees
• NCSL Foundation

• Issue Areas: A-Z
• State-Federal Relations
• NCSL Standing Committees
• News from D.C. and the States

• About State Legislatures
• Elections, Campaigns & Redistricting
• NCSL's StateConnect Directory
• Web Sites

• Online Guide to NCSL Services
• Staff Sections and Networks
• Meetings & Professional Development
• Staff Directories

• Meetings Calendar
• Online Registration
• Meetings Home Page
• Legislative Summit

• Bookstore
• State Legislatures Magazine
• NCSL's StateConnect Directory

NLPES Site Map

2002 NLPES Surveys

NCSL legislative audits database:  As you probably know, NCSL has a searchable electronic database of legislative program evaluation and audit reports (see http://www.ncsl.org/legis/menus/search.cfm).  Do your staff use this database?  Is it a useful way to find other states’ reports on various topics?  Are there any problems with this database or its search capabilities that you would like NCSL to address?

Heather Moritz, Colorado Office of the State Auditor

Our office uses this database but relies more heavily on more general Internet-based searches to identify relevant program evaluation/audit reports from other states (e.g., "Google").  We also use the NLPES listserve to solicit information about other states' evaluation/audit reports when appropriate.  The biggest problem with the NCSL database is you get a number of generic hits that don't give you a lot of information about the document's actual contents.  It is often more expedient to search for a report using other means than wade through the "mystery" information generated by an NCSL-based search.

Generally, our staff have found the NLPES ListServe helpful in getting access to relevant studies and analyses done recently in the various states.  As for the database, some staff have found it fairly useful in getting access to a wide range of information on a particular topic.  Other staff find it difficult to use in terms of sorting substantive work from occasional references.  Time dimensions for the items included are often missing.  One suggestion is to ask every state audit agency to send a notice to NCSL, in a simple but standardized format, every time they release a report.  This notification could be very simple—report title, type of report, a few key words, etc—and not add to each office’s burden.

We find the database useful.  A couple of suggestions: allowing a search on just performance audits (not financial audits), having the title instead of the file name of the report.  Some states' reports have a name like PA200039 instead of the actual title such as "Department of General Services."

Our staff does not frequent the NCSL legislative audits database.  PEER's librarian typically searches federal government, educational, and national organization websites to obtain information for ongoing PEER projects.  The NCSL legisaltive audits database is not as current or "search friendly" as these websites.

Several of our staff indicated that they use the database, including some who do a search each time they start a new project (a wise idea...)  One person said they use the database, but prefer to call the NCSL subject area expert.  Another comment was that the results include a lot of reports that really have nothing to do with the subject searched, but just contain the words in some other context.  For some searches, these search results make it difficult to wade through to figure out which reports are worth looking at.  Also, the way the database pulls the summary information is often not useful.  In some cases, you can't even tell what the report is about from the words that were pulled into the document summary.  So, all in all, we feel it is a good resource to check, but sometimes hard to use.

From time to time someone tries to use the database, but we generally don't rely on it.  We have found listserv inquiries to be the most efficient and efffective way to identify other states' work.  The past indexing of the materials on the database had many problems (for example, our reports are picked up by chapter, resulting in duplicate and unclear references).

Half the New Mexico LFC performance audit staff uses the NCSL web site from time to time but find it difficult to navigate at times.  The biggest frustration is locating the NPLES site.  A direct link to NPLES would be very useful.  Also the same report is often listed twice.

Yes, our staff uses the NCSL database.  Your website is in our procedure manual so that new employees will know where other state audits can be found.  We have found it to be very helpful.  I have not heard of any major problems in using the website.

The database is used by some analysts as they start a new assignment.  However, a number of observations have been offered:

• Standardized descriptions would be helpful to avoid the need to open large PDFs;
• Would be useful to be able to select more than one type of document to search;
• Appear to be inconsistencies in when reports are identified in the search (audit reports not consistently found when a general search is conducted);
• Sometimes just as easy to use the NLPES listserv and post a query, resulting in a personal contact in another state PLUS the relevant report.

Our staff routinely use the NCSL database.  However, because staff are finding that the data on the NCSL database is generally older, they are also going directly to websites of selected states for additional information.  These states are selected either because of their reputation or because our staff are aware of particular projects those states have completed.

I use it some, but I liked the way it used to be better.  Now when I type in a topic I get all kinds of extraneous studies from groups that don't do performance evaluation.  I'd like to see a feature that would allow us to search NLPES member agencies only.

To respond to this question, I surveyed our audit staff (supervisors and above).  Although a number of our staff commented that NCSL’s searchable electronic database sounded like a good resource, the vast majority stated they had been unaware of the existence of this database.  Judging from the number of inquiries we receive through listserv asking if we have issued reports covering various topics, it seems our staff is not alone.  Only two stated that they had used the database and only one of these stated that he used the database recently or regularly.  He stated that he had no luck finding reports on the topics in which he was interested.  However, he could not say whether his lack of luck in finding reports was due to shortcomings in the search capabilities or due to the fact that no one had issued reports on his topics.

The SAO librarian states that our staff use the National State Auditor’s Association (NSAA) database more frequently than the NCSL database.  This is because more search results and information are generated when using the NSAA Audit Report Search Site (http://www.osc.state.ny.us/nsaa/).  The SAO internet page also has a link to the NSAA Audit Report Search Site.  The NCSL site would be more useful if it generated as many search results as are found on the NSAA’s search engine.

We rarely use the audit report database.  When we’re interested in what other states are doing, we send a note via the NLPES listserv requesting that information.

We use it.  We make "check the NSAA database" a standard startup activity for all new audit projects.  We include a link to the site on our internal audit knowledgebase.

Yes, we use the database.  Yes, it is a useful way to find other states' reports.  No, we have not encountered any problems with the database.

Has helped in the planning of our work.  Have not heard of any complaints.

Staff use the database often, but it would be more useful if the search results were produced in a more usable format.  For instance, the search results sometimes do not list document titles and dates the reports were issued.  Also, the search results sometimes list individual report chapters as separate documents, and there’s often not a real summary of the document’s contents.

It’s useful and easy to use.  We look at it at the beginning a project to see if any other states have done anything similar.  Often, there just isn't anything on point.

Our staff occasionally uses the database and hasn't reported any problems with it.

Yes, we use the database at the beginning of each audit to see if our peer agencies have done anything similar to what we are embarking upon.

Had not used it, but I queried it just prior to completing this survey and obtained a large number of apparently off-topic hits.  Perhaps a more valuable resource could be an up to date listing of links to all of the states' on-line audit and evaluation report repositories.

I am aware of it, and have my staff review before we start a project.

We have found the database to be very valuable in planning our projects.  We haven't had any significant problems with the search capabilities.

We have not used this database.  Because its use is limited, we search web sites of performance audit/program evaluation agencies to look for these products.  We would rather ensure completeness of our searches.

Our staff use it and find it useful, although use several other sources as well.  At the NCSL document search site, our staff would like to see the “Select major topics” category expanded because the choices available are so general and don’t encompass many topics.

Staff do use this database.  It is becoming more useful over time.

Some staff have utilized the database and it has been helpful.

Some members of our staff have used this and found it useful.

Occasionally.

N/A

Yes Yes --

Our office has used the database to conduct a study of gifted identification mandates and transportation funding formulas in other states.