The Working Paper



Chair's Corner | Nathalie Molliet-Ribet (Virginia)

Winter marks the start of a new legislative session for many of us, and also the beginning of a new year for NLPES and its Executive Committee members. I have the honor of serving as your new chair; Rachel Hibbard (Hawaii) will serve as vice-chair, and Greg Fugate (Colorado) as secretary. We and the other members of the Executive Committee look forward to serving you in 2016. 

Beginnings are the perfect time to reflect on past performance and future opportunities. The year 2015 was filled with achievements for NLPES, and I hope you personally benefited from many of them. For the first time, NLPES members presented not just one but two webinars, both of which were very well received. The webinar on pivot tables was among the top five NCSL webinars in 2015. Not bad for a “niche” technical topic! In 2015, offices in more states applied for awards and participated in the judging process than in previous years, and the NLPES newsletter was revamped to better align with your interests.

The 2015 Professional Development Seminar in Denver was a great success, with more professional speakers than in years past, and new programming—including a new crowd-pleaser, the poster session. Attendance for the three-day event was the highest since 2006, with 143 people representing 30 states. Many thanks to our colleagues from Colorado, especially Greg Fugate and Vickie Heller, for their hard work, creativity, and hospitality.

Looking ahead, several terrific professional development opportunities are in the works. Be on the lookout for a new webinar this spring, and for NLPES-sponsored training opportunities during the NCSL Legislative Summit in Chicago this summer. Preparations for the fall Professional Development Seminar are underway, to be hosted by our colleagues from Mississippi PEER. We hope to see you in Jackson!

During my term as chair, it is my goal to make available more training and reference materials from member offices through our website. Your offices do great, innovative work, and NLPES is the perfect forum to transfer knowledge across states. If you have ideas or input on other ways to serve you, please do not hesitate to contact me or any member of the Executive Committee!

In addition to using professional development resources and sharing your ideas, there are many ways to get involved in NLPES. If you have not already done so, consider applying for an award to receive recognition for your office or for a specific report; reach out to colleagues through our listserv; submit an article of interest or share office news in the next newsletter; and think about the benefits of a NLPES peer review tailored to your office’s needs. Also, please consider running for a place on the Executive Committee! Four seats are scheduled for election in spring 2016. We welcome your involvement and appreciate your commitment to sustaining NLPES, the only national organization dedicated to advancing legislative program evaluation and auditing.

Wishing you a happy and productive year!

Nathalie Molliet-Ribet is the 2015–2016 NLPES Executive Committee chair.  She can be reached at

Report Radar—Roundup of Evaluations from Across the Country | Angus Maciver (Montana)

image of magnifying glass over paper

The New Year brings new challenges and, to world-weary auditors and evaluators, also some familiar issues. We tireless defenders of government efficiency usually finish the year with a surge of new reports, but also gear-up for another year of ceaseless toil. To aid in our search for productive work, this edition of Report Radar brings you a bunch of excellent reports addressing Veterans Services, Real Property Management, Education, Law and Order, and Emergency Preparedness.

Veterans Services. The country’s growing population of military veterans gets much of its support and services from the federal government, but states continue to play an important role in serving this group. We highlight three reports from member offices looking at how state veterans departments provide services. We begin in December with a report out of Georgia, which addresses the state’s Department of Veteran Services, specifically looking at funding sources and availability of performance data for these functions. Also in December, we had a report on the Department of Veterans Services in Virginia, which discusses options for promoting a more strategic approach to service provision for veterans. In January, the Louisiana Legislative Auditor also released a report on the state’s Department of Veterans Affairs. This report is more investigative in nature and discusses various allegations of misuse of funds and other questionable practices.

Real Property Management.  Four member offices have tackled this subject matter in the last six months or so, beginning in June of 2015 with a report from our people in North Carolina looking at real property portfolio management. This review identified more than $14 million in unused property that could be disposed of, and also discusses problems with data availability and other portfolio management practices. Similar issues surface in an August report from Montana, which identified a lack of a comprehensive inventory of real property and discusses options for reorganizing property management functions within state government. A different take comes from a September report released by the West Virginia Performance Evaluation and Research Division. This report focuses more on the maintenance angle and the state’s ability to properly maintain an overextended real property inventory. Lastly, Michigan has a report from December on the state’s Real Estate Division, which has some good stuff on property leasing activities.

Education. We have a diverse assortment of education-themed reports with a shared focus on performance and measurement. First, another Georgia report, this one looking at math and science salary incentives for K-12 teachers. These incentives were designed to increase recruitment and retention of teachers in these subject areas, but the report notes they are not working as intended. As students move toward college age, an issue that is getting a lot of attention is readiness for post-secondary study. Our friends at the Tennessee Office of Educational Research and Accountability have a great primer on the subject of college readiness, which was released in January. Of course, once students get into college, the focus moves to graduation and, more recently, on-time degree completion. We also have a report on that, courtesy of the New Mexico Legislative Finance Committee, which reported on this subject in December. Finally, once out of college, many of us focus on seeking gainful employment. The question of degree completion and subsequent job placement and, specifically, how university systems measure these statistics is the subject of a November report from the Utah Office of the Legislative Auditor. All these reports are well worth reading prior to your next education-themed project.

Law and Order. We have a mixture of new and more familiar issues in four different reports. Beginning in Florida, we have two reports with the first addressing a new issue that has attracted a lot of recent attention; civil asset forfeiture. This refers to the law enforcement practice of seizing assets that may have been used to conduct criminal activity and the November Florida report discusses reporting requirements and potential options for reforming these practices to limit use by law enforcement agencies. Also from Florida is a December report on a more familiar subject: registration and monitoring of sex offenders. If you have looked at this subject before there will be some familiar issues discussed, including outdated and inconsistent monitoring. With many policymakers focusing on prison overcrowding and attempts to reduce the nation’s incarceration rate, state boards of pardons/parole are one part of the criminal justice system that is attracting interest. If this is the case for you, we encourage you to get a copy of a January report from Utah. The report is a good overview of the state’s Board of Pardons and Parole and includes recommendations addressing oversight, structure, decision making, data collection and business operations.

Emergency Preparedness. Our last offerings for this edition of Report Radar address Emergency Preparedness. The first report was released in October by the California state auditor and looks at emergency preparedness functions in the state’s health department and the governor’s Office. This report is part of a high risk series focusing on these issues in California and details some of the progress that has been made in improving these functions in the different agencies reviewed. Also on emergency preparedness, but this time with a specific focus on railroads, we have an October report from those fine folks in Montana. This report discusses different aspects of the state’s efforts to address railroad safety issues, with a particular emphasis on the transportation of hazardous materials by rail.

Regular readers will know that Report Radar traditionally closes with a miscellaneous selection of great and interesting reports, so not wishing to disappoint, we also commend the following to your attention: Colorado in January for a report on the state’s use of funding for immunization programs; a December report from Connecticut on health information privacy; an October report from New Hampshire on state-owned dams; and two reports on the use of procurement/payment cards in state government from Mississippi and Michigan. Enjoy all these reports to the absolute limit of human happiness, and see you next time on the Report Radar.

Angus Maciver is the deputy legislative auditor for the Montana Legislative Audit Division. He can be reached at

State Profile—Washington's Joint Legislative Audit and Review Committee | Valerie Whitener (Washington)

Picture of Washington JLARC staff in front of capitol

Staff to JLARC provide the Legislature with independent, objective, and credible answers to its questions about government performance. Our vision is to provide answers that are highly regarded and responsive to the Legislature, with work products that are easy to understand, easy to access, and continually improving. We follow yellow book standards, and NLPES coordinated our last three peer reviews.

Bipartisan JLARC Committee has long history
The Legislature designed JLARC to be bipartisan. Committee membership is divided evenly between the House of Representatives and the Senate, and between the two major political parties. Committee members elect an Executive Committee to provide leadership for JLARC, with one member from each of the four political caucuses.  Washington’s Legislature has a long tradition of having its own performance audit function. JLARC (and its predecessor committee) have appointed a nonpartisan legislative auditor since 1951.

JLARC staff are experienced
JLARC’s 21 nonpartisan staff work under the direction of the legislative auditor. Our intrepid research analysts come from a wide variety of educational and work backgrounds. They are generalists who are able to work on assignments that cover nearly every facet of state government. Our support team designs graphics, proofreads our work, helps develop technology solutions, and keeps the office and committee running smoothly.

We are guided by our office’s core values

Trust and respect

  • We act in a way that encourages the development of mutual trust and respect.
  • We respect evaluated agencies by communicating during the audit process and providing opportunities to inform and comment on our work.

Neutrality and objectivity

  • We value the role of the Legislature as the policy-making body and assume a neutral stance regarding the merits of the Legislature’s policies. Our products help the Legislature make informed decisions.
  • We maintain an objective frame of mind about performance of programs and agencies, and base judgments on reliable evidence. We adhere to the standards of our profession.

Growth, collaboration, and supportive workplace

  • We invest in and challenge ourselves. We improve by learning from our colleagues and experiences. We explore and encourage new perspectives and are open to change.
  • We work cooperatively and exchange ideas with colleagues and experts.
  • We embrace humor, creativity, camaraderie, and a balance between work and personal life.

Legislature assigns studies to answer important questions
The Legislature assigns studies to JLARC and establishes due dates through policy bills and budget provisos. The studies address topics that are important to many members. For example, recent reports answered the Legislature’s questions about tax preferences, K-12 health benefits, worker retraining, Medicaid fraud, workers’ compensation, and the feasibility of shifting certain part-time public workers to the state’s health benefit exchange. We present each of our studies to our committee at public meetings, and our reports and presentations are published online.

We are adapting to a new media favored by legislators
We realize more and more legislators are using laptops, tablets, and even their phones to access information. In response, we replaced written reports with a series of web pages for each study. As we implement this new approach, we are learning it’s hard to do well! Still, the response from members and committee staff has been positive, and we know it’s worth the effort. In January, we began announcing report releases via Twitter. You can follow us @WaLegAuditor.

Valerie Whitener is an audit coordinator at JLARC. She can be reached at 

Staff Happenings: Texas Sunset Advisory Commission | Emily Johnson (Texas)

image of Texas capitol

The Texas Sunset Advisory Commission experienced some significant staff changes this year, including the notable retirements of Karl Spock and Ginny McKay, who combined had over 40 years of experience working for the Sunset Commission and the Legislature and who contributed significantly to the field of program evaluation.  We also lost one of our senior managers, Karen Latta, who departed after 18 years to go help implement one of Sunset's recommendations at the Workforce Commission. 

But, with those departures came a talented crop of new staff.  We hired seven new policy analysts and one administrative staff to help conduct the 24 reviews Sunset will undertake over the next two years. 

Emily Johnson is a policy analyst at Sunset. She can be reached at 

Blast from the Past—Performance Auditing in State Government: What's on the Horizon? | Lennis Knighton (Utah Office of the Legislative Auditor General)

image of words "blast from the past"

The following is excerpted from an article written by Lennis Knighton that appeared in the July 1985 issue of the LPES Report (forerunner to the NLPES Working Paper). Knighton was the Utah Legislature’s first auditor general.  He was also one of the original drafters of the GAO Yellow Book and became a professor at Brigham Young University after serving two years as auditor general. It is interesting how Knighton’s observations are still valid 31 years after he penned this article.

The past two decades have been a time of significant development for performance post-audit programs in state government, both in the expansion of operational auditing as well as in the development of program audits and evaluations.  All who have participated in or contributed to these developments can be proud of what has been accomplished, but we must not rest content.  We have struggled from birth through the early years of growing and becoming to reach a level of maturity we might call adolescence, but we must not assume that we now know everything and can do anything.  We still have a long period of learning, growing, and maturing ahead of us.

As I survey the landscape ahead, I see a future bright with opportunity and filled with exciting new developments, but not one devoid of potential pitfalls and perils.  Some of the key trends or conditions that will determine our future are the following:

  • Budgets will continue to be very tight. Consequently, legislative bodies, as well as executives and administrators, will increase their efforts to find ways to improve operational efficiency and to assess program results in terms of both costs and benefits. Budgeting policies and processes will continue to improve, but so too will management control systems and performance evaluation programs. Particularly in legislative review of public programs and expenditures, I foresee a greater use of the work of legislative auditors and program evaluators.
  • Public concern over the size of government will grow. This concern will be fueled by frustrations over the large and continuing federal budget deficits, uneasiness over the mounting government debt and its impact on credit markets and interest rates, and fears of higher taxes. We will hear increasingly that government is too big, that we cannot afford everything we have been doing, that something must be done to cut public expenditures. While there will always be a strong voice from many interest groups for government to do more in certain program areas, the next decade or two is more likely to be dominated by voices calling for reductions, cutbacks, and the elimination of many programs.
  • Public confidence in social programs will continue to fall. Public interest groups, the media, and public officials will increasingly question whether public programs are making any real difference, whether they are having their intended effects, and whether they are worth their cost. Even among liberal groups that have long championed large public expenditures for a wide range of social programs, there is disenchantment and disappointment today over the apparent poor results. And the question everyone will ask in the years ahead is “Why?”  Is the failure of these programs to meet expectations due to a flawed concept, poor design, inadequate funding, poor management, or what? Data from many studies, research projects, and program evaluations will be needed as we sort these issues and seek answers to these questions.
  • Politics will continue as usual. There is widespread hope among most professional auditors and evaluators that the day will come when rational analysis and a careful consideration of the facts will dominate legislative decision-making in the public sector, but it will not happen.  As a people, we do not want it to happen, in spite of popular rhetoric otherwise. We vote people into office not because they are proficient at using their calculator, but because we believe they will champion a set of principles and a concept of government compatible with our own.  We expect them to look after our interests. Legislative auditors and program evaluators must expect and accept this condition as inevitable and essential, and they must learn to work effectively within this environment or their work will largely be ignored.
  • Technological advances will continue to have a significant impact on government and on society. Who can foresee where we are going with computers, telecommunication systems, or scientific and technological breakthroughs in a thousand other fields and disciplines?  As one scientist recently observed, “We are limited only by the speed of light and the imagination of mankind; and maybe not event by the speed of light.”

In summary, while many forces and circumstances will affect our future, our destiny is really in our own hands. We can seize the opportunity and move vigorously to develop our discipline in the decades ahead, or we can allow ourselves to be individually shaped by the winds and currents of our own circumstances.  Our collective response will determine whether we become a mature, professional discipline or merely another interesting episode in a cycle of changes and reforms. We must not rest content.

This is the fifth in a series of ‘oldie but goodie’ articles culled from past issues of the NLPES newsletter.  

Using Excel Conditional Formatting for Improved Reader Comprehension | John Turcotte (North Carolina)

Image of the words "What We Do"

Program evaluators may use the Excel conditional formatting tool to establish rules for highlighting exceptions or patterns in spreadsheet data by using colors, data bars or icons. An example is the dashboard published as a companion to the North Carolina Program Evaluation Division reference document, How North Carolina Compares (HNCC).

HNCC ranks North Carolina relative to other states on areas including population, health, education, taxes, and expenditures. This publication was designed as a quick reference that allows state comparisons across indicators and may be useful when evaluating outcomes of state government programs. 

The Program Evaluation Division has published HNCC in odd-numbered years since 2009 with our most recent edition published in February 2015.

Image of front cover of How North Carolina Compares bookletImage of pages of How North Carolina ComparesHNCC is printed as a pocket booklet the size of a foldable brochure. When opened to one of the indicators, the booklet displays two facing pages.

The left page is a table showing how each state ranks with North Carolina highlighted and all states above average appearing in a shaded block. The right page has a miniature US map with states that performed above the national average darkened followed by bulleted interpretation points. We found that the format lent itself well to quickly looking up one or a few of the 23 HNCC indicators.

However, we learned our General Assembly members wanted a more general picture of how our state compared over time on all of the indicators. It was difficult for anyone to comprehend patterns among 23 indicators for the latest year and new editions were not providing rankings for earlier versions of HNCC. It was also a challenge to do presentations without having to move laboriously through single slides portraying tables displaying North Carolina’s ranking separately for each of the indicators and then verbally explaining what the rankings indicated. 

In 2013, we discovered Excel conditional formatting rules could produce a companion HNCC document—a one page, two-sided dashboard, which uses a green-red color spectrum to portray where and to what degree North Carolina ranks favorably and unfavorably among states year-by-year since 2009.

We rank state indicators in “aspirational” order instead of ranking using values alone. A problem with rankings is that higher nominal values for an indicator may be either good or bad depending upon what is measured. “High” ranks generally imply favorability. However, states do not want to rank high on unemployment rates but instead aspire to have low rates. Interpreting the HNCC dashboard is simple—green is good and red is bad. For example, states aspire to have low unemployment rates, but unfortunately North Carolina currently has a relatively high rate as indicated by our ranking of 39 shaded red-orange—not at the bottom but far enough down in the rankings to be a concern.

Conversely, we are tied with nine other states at a ranking of one with regards to AAA bond ratings and hence the green shade. Rankings on neutral descriptive indicators of interest to our legislators with no aspirational dimensions are unshaded or white, e.g. population and percentage of roads maintained by the state. Note that as North Carolina’s rankings decline, shades change from darker to lighter greens by adding yellow approaching the middle ranks and then blending more red to become orange and then lessening the yellow from the orange until becoming fully red at the bottom where the most unfavorable rankings reside. Excel does this automatically and with high granularity.

Creating the How North Carolina Compares Dashboard

image of a page of How North Carolina Compares
  1. Arrayed North Carolina’s rankings on 23 indicators for each year of HNCC.


  1. Sorted the indicators separately for each year from highest to lowest.


  1. Left mouse button selected (highlighted) rankings for a year as a vertical array. Each year was treated as a separate array for formatting.


  1. Left-clicked the conditional formatting tool on the Home tab.
image of words "Conditional Formatting"
  1. Chose the color scales formatting option. Through trial and error, we found that the choice shown by the arrow did what we intended by shading highest aspired ranks green.
image of the color scales conditional formatting
  1. Added a color scale legend and interpretation guidance notes.



Other Conditional Formats Available Plus Customized Formats Not Shown


High Low

Data Bars

Icon Sets

Image of logic conditional format

Image of High Low conditional formatting

image of data bars conditional formatting

Image of icon sets conditional formatting

John Turcotte is Director of the Program Evaluation Division of North Carolina’s Legislative Services Commission.  He can be reached at

NEWs AND snippets

Call for Executive Committee Candidates!

Image of a check mark in a box

The 2016 NLPES Executive Committee election is almost here! This year, we are electing members for four open positions on the executive committee. Please consider becoming a candidate for one these positions, and submit a brief candidate statement to Brenda Erickson. The statement, limited to 200 words, should include a brief biography and your proposed goals for NLPES. The statement may be included within the body of the email itself or attached as a Word document. The deadline for the submission of candidate statements is March 18, 2016.

Please remember to obtain approval to run for the NLPES Executive Committee from your office’s director or other supervisor, as necessary. Also, serving on the committee requires participation; the committee usually meets three times year—once in the spring, during the NCSL Legislative Summit, and before the NLPES Professional Development Seminar.

Additional information about the NLPES Executive Committee is available on the NLPES website. If you have any questions about this year’s election, please contact Wayne Kidd, NLPES Elections Chair, by phone at (801) 326-1758 or email.

Stop the Presses! Performance Reports in the News | Dale Carlson (California)

Image of person holding newspaper with headline "Extra!!"Based on the large number of articles mentioned below, our offices were certainly busy issuing reports—and getting attention—during the last few months of 2015.

Also, the variety of issues on which our offices reported was really stunning. The report topics the media covered included poor contracting practices, widespread mismanagement, inaccurate revenue collections, budget overruns, program deficits, broken rules, safety weaknesses, and underfunded pension systems.

It looks like we closed out 2015 with a big bang! Let’s keep up our great work as we move into 2016.

CFEC audit recommends major overhaul but no move to Fish and Game
Oct. 22, 2015 – Alaska Dispatch – A legislative audit released this week recommends a sweeping overhaul of the troubled body that handles lucrative limited-permit entries for Alaska’s commercial fishing industry, including a backlog of applications that goes back decades.
Division of Legislative Audit  [Click for full report]

2 ways to reform the Arizona Commerce Authority Dec. 17, 2015 – The Arizona Republic – Excessive crowing has put the state’s most conspicuous economic development organization, the Arizona Commerce Authority, in a spot of trouble. A recent Auditor General report faulted the ACA for boasting about the jobs companies participating in its programs promised and being sketchy about jobs actually created.
Office of the Auditor General  [Click for full report]

Lawmakers question DHS officials on audit
Sept. 28, 2015 – Arkansas News – Arkansas Department of Human Services officials were on the hot seat Friday as members of a legislative panel questioned them about an audit report critical of the agency’s procurement practices. The audit by the state Division of Legislative Audit looked at three DHS projects and found that the agency did not adequately hold vendors accountable for contract fulfillment; used sole-source contracts improperly; did not always use cooperative agreements properly; did not keep the Legislature adequately informed; did not always comply with state purchasing laws and rules; and inadequately managed projects.
Legislative Audit  [Click for full report]

Audit finds widespread mismanagement at Central Basin water district in L.A. County
Dec. 3, 2015 – Los Angeles Times – Poor leadership, a lack of planning and widespread mismanagement have threatened the long-term financial health of the Central Basin Municipal Water District, which serves 2 million people in southeast Los Angeles County, according to a new state audit.
State Auditor  [Click for full report]

Audit: Colorado revenue mistakes cost $7.1 million
Dec. 7, 2015 – 9News – The Colorado Department of Revenue's problems with identifying which taxing districts certain businesses are in led those companies to over-collect $3.3 million for certain local governments and under-collect $3.8 million for other cities, counties and special districts, according to a report released Monday by the Colorado Office of the State Auditor.
Office of the State Auditor  [Click for full report]

Enterprise Florida's overseas sales numbers questioned Dec. 10, 2015 – Orlando Business Journal – The Legislature's accountability office said Enterprise Florida needs to do a better job of separating anticipated overseas sales figures from the actual sales, and has also advised lawmakers that Enterprise Florida should establish goals to assist a wider range of companies starting to get into the export-import field.
Office of Program Policy Analysis & Government Accountability  [Click for full report]

State faulted in Obamacare signup
Dec. 30, 2015 – Honolulu Civil Beat – In a report released Wednesday, the state auditor faulted Hawaii’s Department of Human Services in failing to implement the state’s online system for the Affordable Care Act known as Obamacare. DHS did “not properly plan for or implement” the $155 million health care exchange.
Office of the Auditor   [Click for full report]

Some small gains reported, but state employees still seriously underpaid
Dec. 16, 2015 – The Spokesman-Review – Idaho has made some small gains in the past three years on the percentage of state employees who are paid less than policy rates for their positions, according to a follow-up report from the Legislature’s office of Performance Evaluations, but the state still has a long way to go.
Office of Performance Evaluations  [Click for full report]

Audit claims Kansas social services system $46M over budget
Dec. 21, 2015 – – A recent legislative audit has found that a new computer system designed to make it easier for Kansas residents to apply for Medicaid and other social services is more than two years past due and at least $46 million over budget. The Legislative Post Audit says the Kansas Eligibility Enforcement System won't perform as originally intended when it goes fully online, or produce the projected savings in operational costs.
Legislative Division of Post Audit  [Click for full report]

Audit finds MDOT broke its own rule on road safety studies
Dec. 23, 2015 – WMUK 102.1 – The Michigan Department of Transportation (MDOT) broke its own rule on road safety studies. That’s according to a new report from the Michigan auditor general’s office.
Office of the Auditor General  [Click for full report]

Report questions effectiveness of pilot pre-K programs
Dec. 10, 2015 – Jackson Free Press – Four-year-olds enrolled in the state's special pilot programs for early-learning collaboratives scored no better on kindergarten readiness tests than their peers in other public pre-K programs, a new state report shows.
Joint Committee on Performance Evaluation and Expenditure Review  [Click for full report]

Montana railroads raise safety concerns
Oct. 30, 2015 – KULR News – The Legislative Audit Division has found that Montana’s railroad safety is not up to par. The review concluded that the public service commission is not actively perusing railroad safety with an inadequate number of inspectors to provide coverage for the whole state.
Legislative Audit Division  [Click for full report]

Nevada Highway Patrol to rework bus policy
Dec. 3, 2015 – School Transportation News – The Nevada Legislative Counsel Bureau conducted an audit of the NHP division of the Department of Public Safety and discovered that not all requirements were being followed, specifically the fact that 155 non-bus vehicles went unexamined.
Legislative Counsel Bureau – Audit Division  [Click for full report]

Report details lack of NH food safety inspections
Sept. 15, 2015 – Food Safety News – There are “systemic inadequacies” in New Hampshire’s food safety programs, according to a report released by the state’s Office of Legislative Budget Assistant. The department limited food safety inspections to accredited establishments, leaving other segments of the public food supply insufficiently regulated or unregulated, the report found.
Legislative Budget Assistant Office, Audit Division  [Click for full report]

Hundreds of state workers paid to do nothing but watch TV, wasting $2.7M
Dec. 16, 2015 – New Jersey 101.5 – A report by the state auditor concludes that as the Woodbridge Developmental Center was in the process of closing in 2014, more than 800 direct care employees stayed on the job collecting their full pay with no responsibilities.
Office of the State Auditor  [Click for full report]

Just 14% of New Mexico’s college students graduate on time
Dec. 7, 2015 – Santa Fe New Mexican – New Mexico colleges have increased the number of associate and bachelor’s degrees they’ve awarded by some 15 percent over the past few years, but the state remains stuck at the bottom nationally when it comes to students completing those degrees on time. A report released Monday to the Legislative Finance Committee says New Mexico ranks 49th in the country, with only 14 percent of students in the state earning a bachelor’s degree within four years.
Legislative Finance Committee  [Click for full report]

NC lawmakers to take on long-standing ‘tier’ system for state aid
Dec. 14, 2015 – The News & Observer – North Carolina’s long-used system for rating the economic need of individual counties is often blind to reality and should be repealed, according to a new report that won lawmakers’ interest on Monday. The new report said that system is an unfair blanket approach that misses downtrodden areas within otherwise well-to-do counties and hasn’t resulted in a meaningful flow of grants or credits to communities most in need.
Program Evaluation Division  [Click for full report]

Audit: Oregon unemployment systems showing age
Dec. 29, 2015 – – While computer programs that process Oregon unemployment benefits and taxes handle most transactions accurately, the systems are inflexible, difficult to maintain, and should ultimately be replaced, according to a Secretary of State audit released Tuesday.
Secretary of State Audits Division  [Click for full report]

Report: consolidate centers for people with intellectual disabilities
Dec. 14, 2015 – – Pennsylvania officials should do more to consolidate the five remaining residential centers for individuals with intellectual disabilities, a new legislative study has recommended.
Legislative Budget and Finance Committee  [Click for full report]

Audit finds South Carolina's pension plan underfunded; vulnerable
Dec. 22, 2015 – WJCL22 – A Legislative Audit Council report released Monday says South Carolina’s pension system for public workers is still severely underfunded and vulnerable to severe economic downturns.
Legislative Audit Council  [Click for full report]

AP exam pilot program draws mixed results
Dec. 8, 2015 – The Tennessean – A state pilot program that covered certain high school exam fees has drawn mixed results, according to a recent report.
Offices of Research and Education Accountability  [Click for full report]

State budget woes hurt students with disabilities
Sept. 15, 2015 – Utah Public Radio – In 2014, the Utah State Office of Rehabilitation, which helps people with disabilities find work and live independently, ran a deficit of $4.9 million. According to a September audit from the state Legislative Auditor General, the agency failed to monitor its growing budget, leaving funding for important programs unsustainable.
Office of the Legislative Auditor General  [Click for full report]

Report: changes needed at Virginia veterans agency
Dec. 23, 2015 – Daily Press – The Virginia program assigned to help veterans with mental-health problems faces "a series of major challenges" that range from lack of direction to staff member qualifications, according to a recent legislative report.
Joint Legislative Audit and Review Commission  [Click for full report]

State auditors tell Wisconsin vocational rehabilitation agency to tighten up operations
Dec. 30, 2015 – – Wisconsin’s Division of Vocational Rehabilitation, which helps people with disabilities get and keep jobs, should tighten up its operations, the state Legislative Audit Bureau said in a December audit.
Legislative Audit Bureau  [Click for full report]

Share your coverage with us! If you would like an article highlighted in our next newsletter, send a hyperlink to 

Executive Committee News

Image of City Park Sunrise photo by Greg FugateThe 2015-2016 NLPES Executive Committee met during the NLPES Professional Development Seminar, which was held Oct. 11-14, 2015, in Denver. 

The committee reviewed the status of ongoing subcommittee work and discussed plans for the next Professional Development Seminar happening in Jackson, Miss., in the fall.

Minutes of the Oct. 11, 2015 meeting will be approved during the committee’s spring meeting. Minutes from previous meetings are available online 

2016 NLPES Awards | Rachel Hibbard (Hawaii)

Image of a blue award ribbonSubmissions for the 2016 NLPES awards are fast approaching!  The deadline to apply is Friday, May 13, 2016.  Please visit the NLPES Awards website to review criteria, and consider applying, for the following awards:

  • Outstanding Achievement is presented to one or more individuals who have made outstanding contributions to the field of legislative program evaluation while maintaining high ethical and professional standards.  The 2015 Outstanding Achievement Award was presented to Max Arinder, retired executive director of the Mississippi Joint Committee on Performance Evaluation and Expenditure Review.
  • Excellence in Evaluation is given to one office each year that made significant contributions to the field of legislative program evaluation over the last four years (Jan. 1, 2012-Dec. 31, 2015).  The 2015 Excellence in Evaluation Award was awarded to the New Mexico Legislative Finance Committee.
  • Excellence in Research Methods is awarded to up to three offices that have demonstrated outstanding use of evaluation research methods in a report released during calendar year 2015.  The 2015 Excellence in Research Methods Award was presented to the Florida Office of Program Policy Analysis and Government Accountability.
  • Certificates of Impact are presented to offices that released a report that resulted in a documented impact on public policy within their respective state. There is no limit to the number of offices that may receive an impact certificate, but offices are limited to one submission per year.  Twenty-three offices were presented Certificates of Impact in 2015.

Each award has three judges; remember to provide your submission to all three judges. All of the judges’ information is listed on the NLPES Awards website.  If you have any questions about the application process, please contact Rachel Hibbard, the 2015-2016 vice chair of the Executive Committee and chair of the Awards Subcommittee, at or 808-587-0807

Websites, Training and Other Resources

Image of the words "Check It Out!"NLPES websiteSpend a few moments touring our NLPES website to learn more about NLPES and see what we do.  You’ll find general information about NLPES, including our by-laws, executive committee membership and subcommittees, state contacts, awards, and information on peer reviews.  We also have a training library and resources including past meeting minutes, newsletters, and more.  Check it out!

Image representing emailNLPES listservThe NLPES listserv is an email discussion group for NLPES members.  By sending a message to, you can reach all listserv subscribers simultaneously.  Listserv members can query other states about evaluation work similar to their own projects, receive announcements about performance evaluation reports and job opportunities from other states, and are notified when the latest edition of this newsletter is available!  To join the listserv, send an email to Brenda Erickson, NCSL liaison to NLPES, with the subject “SUBSCRIBE to NLPES Listserv.”  Include your name, job title, audit agency/organization name, mailing address (including city, state, ZIP code), phone number and email address.  A “Welcome” message will be sent to you once you are successfully added to the listserv.  See the listserv link on the NLPES website for additional information on how to post messages and “netiquette” niceties, like not hitting “Reply All” when answering a query posted to the listserv.  

Are you receiving our listserv emails?  Some states’ systems block NLPES listserv emails. If you think you are not receiving our emails, please check your state’s security system and spam filters, and/or contact Brenda Erickson (

Legislative careers websiteKnow someone thinking about pursuing a career with a state legislature?  Point them to the opportunities posted NCSL’s legislative careers website. Job seekers can explore the types of work legislative staffers perform, including performance evaluation, budgeting, fiscal analysis, legal and policy research and opinions, bill drafting, public relations, librarians, building security, and information technology support. Opportunities are posted by states offering positions under Legislative Jobs.  Attracting young people to work as legislative staff will be increasingly important in the coming years. And even though baby boomers make up about a third of the national workforce, nearly half of legislative staff responding to a survey were 50 years old or older. Replacing those staffers will present challenges.  Check out the welcome video, “A Day at the Capitol,” and learn more about the opportunities and rewards of working for state legislatures.  Watch the videos under Career Paths to hear from staffers in different states describing their jobs and the satisfaction they gain from a public service career.  

Image of a stack of booksNLPES’ online training library—Training Products Matrix. Visit our NLPES online training library for a variety of refresher and training materials! There are nearly two dozen resources on planning and scoping, fieldwork, writing and publication, and management topics.  Most are PowerPoint slides; some are narrated; a few are webinars or podcasts. Check them out!

JLARC presentation on Web reportingIn 2014, the state of Washington’s Joint Legislative Audit and Review Committee (JLARC) began issuing audit reports as Web pages rather than PDF documents. Reports are now readily accessible on computers, mobile devices, and tablets. JLARC has received favorable comments on the change from legislators, legislative and executive branch staff, and the public. Writing for the Web requires a change in perspective, in addition to changes in technology. Visit JLARC’s website to see samples of their Web reports. Contact Valerie Whitener, Audit Coordinator, JLARC,, with questions.

Image for "Ask GAO Live"Ask GAO LiveAskGAOLive is a 30-minute interface where GAO staff chat about a specific report and research, and answer questions that are emailed or tweeted in. Sessions are recorded and archived on the website. You can also “follow” GAOLive to receive advance notice of chat sessions. Topics include veterans and higher education, prescription drug shortages, prison overcrowding, state and local fiscal outlook, and government contracting.

Image of capitol domeEnsuring the Public TrustWhat’s the most common internal performance measure for evaluation shops? How many offices tweet? What percentage of staff has fewer than 10 years of experience? How can you contact a sister office in another state? Ensuring the Public Trust summarizes information about legislative offices conducting program evaluations, policy analyses, and performance audits across the country.


The Working Paper is published three times a year by the National Legislative Program Evaluation Society, a staff section of the National Conference of State Legislatures. NLPES serves the professionals of state legislative agencies engaged in government program evaluation. The purposes of NLPES are to promote the art and science of legislative program evaluation; to enhance professionalism and training in legislative program evaluation; and to promote the exchange of ideas and information about legislative program evaluation.

The Working Paper is produced by the NLPES Communications Subcommittee.
Dale Carlson, 2015-2016 chair (CA)
Shunti Taylor, newsletter editor (GA)

NCSL Liaison to NLPES
Brenda Erickson, 303-856-1391
NCSL Denver Office • 303-364-7700