Picture of NLPES Logo

NLPES November Newsletter banner


November 2012


Picture of Capitol DomeChair’s Corner
Karl Spock (Texas)

Autumn marks the start of a new year for NLPES and its executive committee.  The committee’s officers just changed this October, with me taking office as chair, Lisa Kieffer (Georgia) as vice chair, and Wayne Kidd (Utah) as secretary.  We and the other members of the executive committee look forward to serving you in the year to come. 

Beginnings are good points to reflect on the past and look to the future.  I was looking back in our NLPES newsletter archive and noticed James Barber’s “Chair’s Corner” from April 2010.  Commenting on states’ serious budget difficulties and resulting travel bans and cutbacks, James noted that the “good times have come and gone, but surely will return soon.”

We haven’t made it back to the really good times yet. Some on the executive committee have had to adjust to reduced or nonexistent travel funds, either paying their own way to committee meetings or attending via conference calls. Many states have had difficulty in coming up with funds to send NLPES members to our professional development seminars. Thankfully, NCSL has helped bridge the travel gap with money for webinars and other distance-learning efforts. Also, we very much appreciate the assistance from the Pew Foundation, which has provided stipends for members to attend the last two professional development seminars in 2011 and 2012. We all have continued on.

Signs are out there now that conditions are starting to improve. Ninety of us from 24 states attended the 2011 professional development seminar in Denver. Numbers are up some for the 2012 professional development seminar we just finished in Atlanta, attended by 111 NLPES members from 28 states.  Many thanks to our Georgia colleagues for putting on a great program and engaging us with their Southern hospitality. Presentations from the professional development seminar should soon be online on the training resources page of our NLPES website.

Looking forward, our committee’s Professional Development Subcommittee, headed by Angus Maciver (Montana), will continue to look for webinar and other online training opportunities for you, as well as training opportunities at the NCSL Legislative Summit coming up in Atlanta in August 2013.  Dale Carlson (California), chair of the Communications Subcommittee, will be leading work on the newsletter, website and other communications issues.  Wayne Kidd (Utah) will be chairing our Awards Committee this year.  If you have ideas or input on those topics, please feel free to contact them.

Finally, we here in Texas are delighted to invite all of you to attend next year’s professional development seminar, to be held in Austin in the Fall 2013.  We have already started working on the event, and are looking forward to seeing as many of you as can make it.  Let’s hope budgets continue to improve!  –Karl Spock

Report Radar
Angus Maciver (Montana)

Happy fall all! With the end of the year fast approaching, it’s time to harvest another bumper crop of audit and evaluation reports.  All these products are high in accountability-based nutritional value and will sustain you through the hard winter months.  “What’s on the menu?” we hear you ask:  land conservation, taxation, economic development, health and human services and corrections.

Let’s begin with land conservation. We note two reports released in September addressing states’ efforts to conserve agricultural land or open space. Colorado has a report addressing use of tax credits for conservation easements and highlights issues relating to agency oversight of these incentives for conserving land. The Virginia Joint Legislative Audit and Review Commission took a broader look at the revenue sources available for land conservation, but also included review of available tax credits.  Both are good reads and very informative on the subject matter.

For taxation issues we start in the Evergreen State, where the Washington Joint Legislative Audit and Review Committee released a report in July containing the latest installment of its ongoing work addressing tax preferences. This report summarizes the tax preference performance reviews conducted by Washington in 2012. Our people in Utah have also been looking at taxes and in September released a very interesting report on the state’s radioactive waste facility tax. The Utah performance audit addressed how taxes on companies handling radioactive waste are administered and policy options for revising the tax structure.

Economic development policies and programs were the focus of reports from three different states over recent months. We begin with a June report issued by the Wisconsin Legislative Audit Bureau, which took a broad-based look at all of the Badger state’s economic development programs. The report identifies 196 different programs responsible for $226 million in biennial expenditures for various economic development efforts. Next up is the Maryland Office of the Legislative Auditor, which released a report in October on the Maryland Technology Development Corporation. This program provides start-up financing for technology businesses, works with the state’s research universities on new technologies, and promotes state-funded stem cell research. We finish in New Mexico with an evaluation report released in August that addressed three state programs designed to promote economic development through job creation incentives.

As we would typically expect, many states have reported recently on issues relating to healthcare and human services programs. October saw the release of an audit from the Illinois Office of the Auditor General looking at the state’s health insurance program providing coverage for children (probably similar to federally funded children’s health insurance programs or Medicaid expansions in your state). Medicaid coverage was also the focus of a July report from South Carolina, which looked at Medicaid managed care rates, expenditures and other administrative issues. The Nevada Legislative Auditor issued a report in September that looked at benefits payments made using an Electronic Benefits Transfer card system (identifying the problem of payments being made to deceased individuals). We also draw your attention to a July report from our friends in Maine, which addressed the state’s Child Development Services program and made multiple recommendations to improve program services and outcomes.

We round out this month’s menu with reports addressing corrections issues.  A September report from the Michigan Office of the Auditor General looked at safety and security issues at one of the state’s correctional facilities. September also saw the release of performance audit from Tennessee addressing the state’s Board of Probation and Parole.  The Kansas Legislative Division of Post Audit issued a report in July, which looked at operations in one of the state’s juvenile correctional facilities. Finally, we commend to your attention a report from West Virginia on the state’s approach to regulation of the bail bonding industry.  Bail bonding is a policy issue that has been getting some traction recently and we are unsurprised to find one of our member offices out in front leading the way.  Well done (the other) Mountain State!

By the time you get done reading these wonderful reports Christmas will be upon us. Report Radar suggests re-gifting these products, so your friends and family can also enjoy the benefits of insightful and informative legislative audit and evaluation reporting.  See you next year!

NCSL Legislative Summit and NLPES PDS Provided Learning and Networking Opportunities
Rachel Hibbard (Hawaii)

NLPES at the NCSL Summit in Chicago (August 2012)

Chicago skylineThe National Conference of State Legislatures held its annual Legislative Summit Aug. 6-9, 2012, in Chicago, Illinois.  Legislators and staff alike had the chance to network, learn, and share with peers.  More than 100 informational sessions took place over four days, along with social and networking events.  NLPES co-sponsored four sessions during the Summit:

NLPES joined the NCSL Legislative Effectiveness Committee in sponsoring “Using the Audit Tool:  A Critical Link to Effectiveness,” which looked at  

audit and program evaluation organizations.  NLPES executive committee member Greg Fugate (Colorado) and former executive committee chair Ken Levine (Texas) served as panelists.

We teamed with the Legal Services Staff Section and the National Association of Legislative Fiscal Offices (NALFO) in sponsoring “Legislative Oversight of Federal Funds,” to explore general and legal roles of state legislatures in the federal grant-in-aid system.  Moderated by NLPES member Holly Lyons (Iowa), and featured NLPES’ers Eric Weeks and Jonathan Ball (Utah) as panelists.

We linked with NALFO in sponsoring “Studying Retirement” which addressed Virginia’s process for studying and evaluating its retirement system. Moderated by NLPES member Stephen Klein (Vermont), it included NLPES’ers Tracey Smith and Drew Dickinson(Virginia) as speakers.

We also joined with NALFO in sponsoring “Results First—Targeting Resources at Programs That Work,” highlighting a budgeting approach enabling legislators to identify and compare programs’ return on investment with taxpayer dollars spent.  Panelists included NLPES executive committee member Charles Sallee (New Mexico).

The NLPES executive committee met briefly to approve meeting minutes from Denver (May 2012) and discuss final details for October’s professional development seminar in Atlanta; the NLPES fall newsletter; location of 2013 PDS (Austin, Texas); and vote on amendments to bylaw Article VII (passed).

NLPES also hosted a Dutch treat dinner for its members at the well-known Berghoff restaurant and a networking breakfast midweek, during which outgoing executive committee chair Scott Sager (Wisconsin) handed over the reins to incoming chair Karl Spock (Texas).

NLPES’ annual Professional Development Seminar in Atlanta (October 2012)
NLPES’ 2012 professional development seminar was hosted in Southern style at the Georgian Terrace Hotel in Atlanta on Oct. 1-3, 2012 by Georgia’s Department of Audits and Accounts.  More than 100 people from 28 states participated in this year’s event.Picture of Georgia Capitol

The conference featured three plenary sessions, the highlights of which were presentations by the Pew Center on the States on its Results First program and a roundtable on auditing the politically sensitive.  Over the three days, attendees had a choice of concurrently held sessions, including topical discussions on report writing, interviewing and managing difficult auditees, audit planning, research methods, auditing standards, reconciling agency data, and human resources for audit shops.  Specifics of actual reports were covered in other concurrent panels on education, cost savings, prison alternatives, and veterans’ programs.

 The conference was also a great opportunity to network and shed the feeling of being all alone in the evaluation profession.  Georgia staff went out of their way to make everyone welcome at an evening reception in the hotel’s grand ballroom, at an awards luncheon, and at an evening excursion to a local tavern where we were treated to all manner of Southern specialties, including fried pickles (which, by the way, are alarmingly “more”-ish).

Picture of NLPES members at 2012 Fall SeminarThe success of the conference was reflected in the positive evaluations received from attendees.  Our thanks to Georgia State Auditor Greg Griffin and especially to Lisa Kieffer, deputy director of the Performance Audit Operations Division, and the performance audit staff for all their hard work and hospitality in hosting a very successful conference.  We hope to see you all in Austin, Texas, for our Fall 2013 PDS!

If you missed the Atlanta conference, or just want to revisit some of the great slides from the sessions, presentations will be posted on the NLPES website in the near future. 

State Profile—Wisconsin
Legislative Audit Bureau
Year Established
The Legislative Audit Bureau was created in 1966. Prior to the creation of the Audit Bureau, financial audits were performed by the Department of State Audit, an executive branch department created in 1947.

Governing Body and Duties
The Joint Legislative Audit Committee has advisory responsibilities for the Audit Bureau. The committee may direct the Audit Bureau to conduct audits and evaluations, receives and reviews reports issued by the Bureau, conducts hearings on Audit Bureau reports, and may introduce legislation pertaining to Audit Bureau recommendations.

Audit Work
The Legislative Audit Bureau completes both financial audits and program evaluations.  Since 2008, the bureau has also operated a fraud, waste and mismanagement hotline that allows the public and individuals within state government to report suspected fraud and other improper acts by state agencies, employees, and contractors.

Number and Background of Staff
The bureau is led by an appointed state auditor who serves at the pleasure of the legislative leadership. Currently, the bureau employees 23 program evaluators who typically have graduate degrees in public policy, public administration, political science, economics, business administration, urban and regional planning, or law. The bureau also employs 48 financial auditors who typically have degrees in accounting and have, or are working toward, their Certified Public Accountant certification.

Recent Performance Audits and Studies
In addition to its other evaluations, the Legislative Audit Bureau has published comprehensive evaluations of the State’s Supplemental Nutritional Assistance Program, known as FoodShare, and the economic development programs of eight state agencies.

Other Items
Five things we want the rest of NLPES to know about the Legislative Audit Bureau:

  1. The bureau hosted the NLPES fall training conference in 1996 and 2008.  The corn maze at Eugster’s Farm in 2008 is an experience bureau staff members still reminisce about. (Auditor’s Note: According to past NLPES chair Scott Sager, conference attendees from around the country still talk about that corn maze.)
  2. Some of the bureau’s most unique and high-profile audits have been related to entities ending in “ers,” such as the Packers, the Brewers and the Badgers.
  3. 2012 marked the 20th anniversary of the Legislative Audit Bureau’s annual chili cook-off!  Awards include Blandest Chili, Hottest Chili, Most Original Chili and special awards for overall favorite chili, which are the Judge’s and People’s Choice awards.
  4. Annually, the bureau establishes itself as No. 1 in charitable giving for an agency of its size through the state of Wisconsin’s Partners in Giving campaign.  Recent themes and events to encourage giving in the office have included L.A.B.opoly (a play on the all-time favorite board game), Wisconsin Sports Teams, Head of the Class Penny Wars and the Olympic Gold Challenge.
  5. At the Legislative Audit Bureau’s annual holiday party, staff members nominate other staff members for “Labbies” and share amusing stories related to their audit work.  The prestigious Labbie—the Snowman award—is then presented to the staff member who presents the best Labbie nomination. Inspired by the Sylvester Stallone movie Copland, a past winner was a story about an upstart supervisor in a small Wisconsin audit bureau with a passion for truth and a nose for fraud…to be continued.

Staff Spotlight
(Ed. note:  Staff Spotlight is an ongoing series, written by staff who are relatively new to performance evaluations.  We believe these staff members all have unique stories that describe how they joined the profession and what they like about it so far.  We also believe that others in the profession would enjoy these stories.  If you would like to submit your own story, please send it to Brenda Erickson, NCSL Liaison to NLPES.)

Spotlight I
Candace Ware (Utah)

How many of you can say that you had never heard of the office in which you work before you applied for a position?  For me, this actually was the case.  So, what brought me to the Utah Office of the Legislative Auditor General?  In 2009, I was soon to graduate with an economics degree (now also a Master of Public Policy) from the University of Utah and was looking for a career. I had responded to countless federal career and internship announcements thinking that I would most likely find myself working for the Bureau of Labor Statistics or perhaps as an analyst with the Department of Defense.  The economy and job opportunities in 2009, however, were souring quickly because of the bust of the housing market, and I was not sure how my job search would play out. 

During this time, I also received an email from my university’s economics department indicating there would be an evening meeting to discuss jobs in the economics field. I attended the meeting, but it was not quite what I expected. Instead of a broad overview of the types of jobs economic graduates might enjoy, two performance auditors from the Office of the Legislative Auditor General discussed with us why their degrees in economics helped prepare them for their careers. Although I was not a political or government guru, I had previously heard of the State Auditor’s office which conducts financial reviews. However, up to this point, I was unaware of the existence of the Legislative Auditor’s office and the field of performance auditing. Yet, during the auditors’ presentation, I found myself interested in the type and variety of work the office performed. I remember thinking to myself that it would be great to work in an environment that strives to improve the function and accountability of government while also maintaining independence from political pressures.

In short, I was hired as an audit intern and was assigned a full position as a performance auditor after a year. As of today, I am a senior performance auditor that has worked on a variety of government reviews over the past three years, including topics such as: Medicaid, charter schools, state parks, public transportation, radiation control and special service districts. As with most other performance auditors I know, my favorite part of the work I do is how different one audit is from the next.  It is difficult to get bored on the job when audit subject matter changes once or twice a year. I find it fascinating that I have the opportunity to learn the inner workings of government agencies, to research relevant best practices, and to provide recommendations for improvement and greater accountability.  I am also very grateful for the supportive nature of my colleagues. I find it invaluable to be able to bounce ideas off my teammates as we develop audit methods and tests for a given situation. Overall, my experience with the Office of the Legislative Auditor General has been extremely positive, and I look forward to the years ahead.  –Candace Ware, Senior Performance Auditor, Utah Office of the Legislative Auditor General

Spotlight II
Katie Francis (Virginia)

“If you could be a kitchen utensil, what would you be?”  Little did I know that the answer to this interview question would ultimately place me on a path to become a legislative analyst at Virginia’s Joint Legislative Audit and Review Commission (JLARC).  After blurting out “a spork” as my answer, I received an enthusiastic “yeah!” and a high-five from the assistant director of Constituent Services, and was offered my first full-time job at the governor’s office.

I first heard about JLARC at a career workshop during my senior year in college. I met a former JLARC analyst, Trish Bishop, who discussed the organization’s work in evaluating a myriad of Virginia policies and programs through a rigorous research process. By the end of our conversation, I was sold on working for JLARC. Unfortunately, I was not yet qualified to apply, but I still wanted to work in state government. Luckily, one interview and healthy spork conversation later, I joined the governor’s office of Virginia.

During my time at the governor’s office, I had the opportunity to interact with a multitude of state agencies and learn about key policy issues facing Virginia. I enjoyed the job.  However, working in this role confirmed my interest in serving the state through research and analysis. I left the executive branch to research psychological rehabilitation issues at Virginia Commonwealth University. While I loved the research process and producing academic manuscripts, I missed working on state policy issues and wondered what career would allow me to fuse my passion for research with my passion for studying government policies and programs.

My mind kept coming back to JLARC, but I knew that without more training, a career in policy and program evaluation would remain out of reach. I decided to obtain an MPA before making another career switch. After graduate school, I saw a job opening at JLARC and eagerly applied. I was unsure if I would qualify for the job given my inexperience in the field, so I was very happy that the management staff saw me as a good fit for the agency. I joined JLARC in 2011, three years after first learning about the agency at the career workshop.

Now at JLARC, I’m delving deep into analyzing Virginia policies and programs. Although there are several reasons why I love my job, three stand out: 1) the breadth of issues we evaluate; 2) the opportunity for continuous learning; and 3) the supportive, motivating work environment.  After just over a year here, I have had the opportunity to evaluate the effectiveness of year-round schools, and am currently on a team that is assessing Virginia’s homeland security and emergency preparedness efforts.  Furthermore, JLARC’s research process allows for the continuous development of my analytical skills and issue-specific knowledge. I’m able to conduct in-depth analyses of state issues in a manner that both enhances my knowledge of a particular policy area and produces meaningful recommendations for the legislature, which is one of the most rewarding parts of the job. More important, though, is the opportunity to work alongside such intelligent, experienced professionals whose institutional knowledge and guidance are invaluable to my success as an analyst and the success of our policy recommendations. My colleagues consistently provide me with the support I need to produce quality research throughout the course of a study. In this supportive environment, I can step out of my comfort zone and conduct research that challenges, and ultimately strengthens, my current skill set.

The road to JLARC was long, but every step along the way fueled my passion for research and analysis and made me yearn for a chance to put those skills into practice, making the job that much more rewarding.  Working for JLARC motivates me to truly understand the inner workings of policies and programs in order to positively impact the public, which is incredibly empowering.  I can say, without hesitation or reservation, that my experience at JLARC thus far has solidified my desire to pursue a lifelong career in program evaluation and public service.  –Katie Francis, Legislative Analyst, Virginia Joint Legislative Audit and Review Commission


NLPES Website
Dale Carlson (California)

Want to learn more about the NLPES?  Want to see what we do?  Then spend a few moments touring our website.

It’s easy to get to.  Simply plug “NLPES” into Google, Bing, Yahoo, Dogpile (or whatever search tool you use) and chances are the top one or two results will get you to our site. Alternatively, go to NCSL. Once there, scroll over “Legislative Staff” in the tabs at the top and click “Program Evaluation” in the pop-up window (it’s in the far right column, third from the top).  You’re there.

Once there, wander around to see what the site offers. You’ll find general information about the NLPES (e.g., by-laws, the executive committee membership, etc.), recent or upcoming events, prior newsletters, training resources, awards, NLPES’s peer review program, etc. You can even word search our site looking for specifics. 

Come on by and visit for a while! 


NLPES Listserv
Dale Carlson (California)

Would you like our NLPES Newsletter delivered directly to your email box? Do you want to query other states about whether they’ve done evaluation work similar to work you’ve just been assigned? How would you like to receive email announcements about performance evaluation reports that other states recently issued?

Then the NLPES Listserv is meant for you.

It’s easy to join. Send an e-mail message to NLPES, leaving the subject area blank and including the word SUBSCRIBE in the message area. If successful, you’ll receive a "Welcome" message.  (Be patient about the response, please; it may take a little bit.)  You’ll be glad you joined.

See the listserv link on the NLPES website for additional information (e.g., how to post messages on the listserv, “netiquette” niceties, and so forth.) 

Legislative Careers Website
Dale Carlson (California)

Know a young professional who is thinking about pursuing a new career?  Perhaps a career working for a state legislature?  Point them to the opportunities posted on NCSL’s Legislative Careers website. At this site, job seekers can explore the various types of work legislative staffers perform, including performance evaluation, fiscal, legal, research and security, and can find opportunities listed by various states offering positions.

NCSL launched the website in June 2012. The statement announcing the website pointed out, “Attracting young people to work as legislative staff will be increasingly important in the coming years.”  It also mentioned that even though baby boomers make up about one-third of the national workforce, nearly half of the legislative staff responding to a survey were 50 years old or older. Replacing legislative staffers who leave will present challenges.

From the site’s home page, the “Welcome” tab includes a relatively short video titled, “A Day at the Capitol” and invites visitors to “learn more about the opportunities and rewards of working for state legislatures.”  The “Career Paths” tab includes links to videos of staffers who play different roles in different states describing their jobs and the satisfaction they take in a public service career. The final tab, “Legislative Jobs,” takes visitors to a list of multiple states and links to legislative career opportunities in those states.

Check it out. 

NLPES Executive Committee Goings and Comings
Dale Carlson (California)
The NLPES Executive Committee (EC) recently underwent some changes, losing two of its members to other organizations. Scott Sager (Wisconsin) left the EC in October 2012 bound for the private sector where he will manage consulting engagements with state and local governments, higher-education institutions, and nonprofits. Scott joined the EC in 2007, served as its chair for 2011-12, and most recently was the immediate past chair. Carol Ripple (North Carolina) also left the EC in October. Starting in November 2012, Carol will be a senior research scientist in Duke’s Sanford School of Public Policy. Carol joined the EC in 2011 and served on the Professional Development subcommittee. Best of luck to you both!

Although the EC opted to leave the immediate past chair position vacant (it’s a nonvoting position), we are pleased to announce that Katrin Osterhaus (Kansas) will finish out Carol’s term on the EC, which expires in 2014.  Katrin is a principal auditor with Kansas Legislative Division of Post Audit. Welcome Katrin! 

Mark Your Calendars
Brenda Erickson (NCSL)

Y’all know that Georgia has peaches. But what else does Georgia have?  The 2013 NCSL Legislative Summit, that’s what!  Georgia will host the Summit in Atlanta, Aug. 12-15, 2013. Details about the meeting are not yet available, but watch the NLPES homepage and NLPES News!

The Next NLPES Newsletter
Dale Carlson (California)
Look for the Spring 2013 edition of the NLPES Newsletter next April or May! 

NLPES-NEWS is published by the National Legislative Program Evaluation Society, a staff section of the National Conference of State Legislatures.  NLPES serves the professionals of state legislative agencies engaged in government program evaluation.  The purposes of NLPES are to promote the art and science of legislative program evaluation; to enhance professionalism and training in legislative program evaluation; and to promote the exchange of ideas and information about legislative program evaluation.


2011-2012 NLPES Communications Subcommittee:

Dale Carlson (CA)
Charles Sallee (NM)
Karl Spock (TX)
NLPES website

NCSL Liaison to NLPES: 
Brenda Erickson
National Conference of State Legislatures • 7700 East First Place • Denver, Colorado 80230 • 303-364-7700