NLPES Question of the Month

The NLPES Executive Committee would like to have information on the NLPES web site that highlights good practices, helps members to improve their day-to-day work, and encourages professional development. The Question of the Month is one way to accomplish this.  Each month, NLPES listserve members will receive a "Question of the Month" via e-mail, and members will be invited to submit responses.   You can submit your responses to .  If you have suggestions for a new Question of the Month, contact Joel Alter, Minnesota Office of the Legislative Auditor (651/296-8313). 

Questions of the Month:

Responses to Past Questions of the Month
Topic Question Date
REPORT RELEASE PRACTICES What are your agency´s practices for releasing program evaluation reports to the press and public?  September-October 2006
RECENT STAFFING TRENDS What were your office's number of permanent, full-time staff in the most recent three years (2004-05, 2005-06, and 2006-07) in each of the following areas:  (a) program evaluation/ performance auditing, (b) financial auditing, and (c) support staff. June-August 2006
CONTRACTING FOR PROGRAM EVALUATIONS Has your office contracted out any of its performance audit work to private firms? (Why or why not?  If you have contracted, what types of work have you contracted for?  How have you identified potential bidders for RFPs? How have you monitored the work and workpapers of  the contractors?) June-August 2006
INTEREST IN WEBCAST TRAINING What would be the potential benefits if ncsl were to produce "webcasts" or "podcasts" for your staff's professional development, and what concerns/questions about this approach do you have? March/April 2006
FOLLOW-UP ON PAST REPORTS How does your agency follow up on the recommendations of previous reports?  January-February 2006
QUALITY CONTROL PROCEDURES What quality control procedures does your office use to ensure the accuracy of your reports?  October-December 2005
EVALUATING CONTRACTING PRACTICES Has your office developed standards for evaluating state agencies' contract management practices? July-September 2005
USE OF GIS FOR EVALUATION PURPOSES How can geographic information system (GIS) tools be used in program evaluation work?  May-June 2005
CHALLENGES TO DATA ACCESS How have you addressed situations where your access to evaluation data has been challenged? April-June 2005
INTERVIEW QUESTIONS FOR JOB APPLICANTS When hiring new staff, what are some examples of interview questions that have been effective in helping you identify great candidates?  January March 2005
CONFIDENTIALITY OF SURVEY RESPONSES When conducting surveys, do you promise confidentiality to potential respondents? (If so, in what circumstances do you do so? And how do you word your pledge of confidentiality? If you don't promise confidentiality, why don't you do this?)  January-February 2005
DEALING WITH THE PRESS What suggestions do you have for dealing with the press?  (What bad experiences have you had, and what lessons have you learned?  Do you have suggestions for ensuring that the press accurately conveys the messages in your reports?) October-December 2004
ONLINE SURVEYS What’s been your experience with online surveys?  (Have you conducted surveys online?  If so, what lessons have you learned?  If not, why haven’t you done so, and would you consider doing online surveys?  Do you have any observations about the components of a well-designed online survey? July-August 2004 
WORKPAPER CONFIDENTIALITY How do you ensure that sensitive information collected for your evaluations is not exposed to public view?  (Does state law allow you to keep certain work papers confidential?  Do you redact documents that you think should not be available for public review?  How do you determine which work papers will not be public?) April-June 2004
REVIEWING AUDITEES' E-MAILS OR HARD DRIVES "Do you have tips for accessing auditees' e-mails or hard drives, in cases where review of this information is necessary?" February-March 2004
FINANCIAL AUDITORS IN EVALUATION PROCESS How, if at all, does your program evaluation staff use the expertise of financial auditors during the evaluation process?  November-December 2003
MINORITY RECRUITMENT Do you have any suggestions for ways that nlpes offices could improve the recruitment of minorities for program evaluation positions?  August-October 2003 
WRITING EXERCISES FOR JOB APPLICANTS What's your opinion about requiring job applicants to complete a writing exam or writing exercise?  August-October 2003 
IMPACT OF STATE BUDGET CUTS Have state budget problems had a direct effect on your office or day-to-day work this year-and what specific impacts do you anticipate in coming months?  May-July 2003
POLICIES FOR DISTRIBUTING HARD COPIES OF REPORTS What are your office's policies for distributing hard-copy versions of your reports? March/April 2003
AGENCY WEB SITES How has your agency's web site improved your work and your ability to serve the public? January/February 2003
WORKPAPER RETENTION How does your office retain its workpapers?  October/November 2002
TAPE RECORDING AUDIT-RELATED INTERVIEWS Do you ever tape record interviews during your program evaluations/audits--and, if so, in what circumstances? August/September 2002 
DESKTOP PUBLISHING SOFTWARE Based on your agency's experience, what are the pros and 
Cons of various desktop publishing programs  (such as Microsoft Publisher, Adobe Pagemaker, Ventura, etc.)?
June/July 2002
COST SAVINGS REPORTS How, if at all, does your office report estimates of cost Savings that have resulted from your recommendations? May/July 2002
IMPROVING RELATIONSHIPS WITH AGENCIES EVALUATED What steps does your office take to improve working relationships with the agencies you're evaluating? March/April 2002
REPORT FORMAT What do your office's written reports look like, and what are you trying to accomplish with your format(s)? January/February 2002
INTERN USE Does your office have internships--and, if so, how have you incorporated interns into your evaluation and performance audit work? November/December 2001
RESPONDING TO POLICY QUESTIONS When your legislators ask you, as an audit or program evaluation office, for answers to policy questions, how do you respond? October 2001
WORK ATTRACTIONS & CHALLENGES Based on your experience in legislative program evaluation so far, what are the most significant attractions and the most difficult challenges of your daily work? (This is intended for staff who have been working in a legislative program evaluation office for 5 years or less) August/September 2001
FLEXIBLE WORK OPTIONS What flexible work schedule options does your office offer, and how important have they been in helping you attract and retain employees? July 2001
INTERNET RESEARCH TOOLS What tools or resources for program evaluation research have you found on the Internet that might be useful to colleagues in other states? May/June 2001
HIRING PRACTICES What approaches has your office used to screen and hire the best candidates for employment as evaluators? April 2001
EVALUATIONS OF THE JUDICIAL BRANCH What role, if any, has your office had in evaluating services provided by your state's judicial branch of government, and are there legal or other impediments to doing such studies in your state? February 2001
TIMELY COMPLETION OF PROJECTS What steps does your office take to ensure that projects finish on time? January 2001
TRAITS OF AN OUTSTANDING PROGRAM EVALUATOR Beyond the skills acquired through formal education, what are the most important traits of an outstanding program evaluator (or performance auditor)? September 2000
WEBPAGE IMPACT ON REPORT WRITING, DISTRIBUTION How, if at all, has the development of your agency's web page changed the way that you write reports, distribute reports, or present information to your audience? August 2000
WAYS TO IMPROVE SURVEY RESPONSE RATES What approaches has your office used to try to increase response rates for surveys--and do you have any benchmarks for minimally-acceptable survey response rates? June 2000
WAYS TO IMPROVE TEAMWORK AND COLLEGIALITY What interesting approach (or approaches) has your office used to improve teamwork and collegiality among staff? May 2000