2017 NLPES Professional Development Seminar Preliminary Agenda

NLPES Executive Committee Meeting

Madison Concourse, 2nd Floor, Conference III

Presiding: Linda Triplett, Mississippi PEER Committee and Incoming NLPES Chair

4-6:30 p.m.

Registration
Madison Concurse, 2nd Floor Foyer

Special Optional "Dutch Treat" Event

Preregistration for the lake cruise was required.

• Walk from hotel to landing at 6 p.m.
• Board boat at 6:15-6:30 p.m.
• Cruise is from 6:30-8:30 p.m.

Registration  

Welcome and Opening Remarks

State Capitol, Assembly Chamber

Welcomes from the Wisconsin Host State, the NCSL Staff Chair and NLPES leadership.

Speakers: Joe Chrisman, Wisconsin Legislative Audit Bureau

Charles Truesdell, Kentucky Legislative Research Commission and NCSL Staff Chair

Plenary Session 1—The Value of Evaluation: The Legislative Perspective
State Capitol, Assembly Chamber
This session will feature remarks from several Wisconsin legislators about the important role of nonpartisan staff and the value of evaluation in policymaking.

Moderator: Joe Chrisman, Wisconsin Legislative Audit Bureau

Panelists: Wisconsin legislators

Tour of the State Capitol  

State Capitol

Return to Hotel

NLPES Awards Luncheon
Madison Concurse, 2nd Floor, Wisconsin Ballroom

Presiding: Linda Triplett, Mississippi PEER Committee and incoming NLPES Chair

Concurrent Session 1—Evaluating Economic Development Programs: Lessons Learned for Getting Results
Madison Concourse, 2nd Floor, Madison Ballroom

From tax credits to promote job creation and from loans to spur capital investment, states are using a variety of tools to encourage development in the private sector. In this session, we will examine the methodologies used to assess the outcomes of state investments in economic development and review strategies for assessing the effectiveness of economic development programs, particularly when definitive outcome-related information is unavailable or unreliable.

Moderator: Kate Shiroff, Colorado Office of the State Auditor

Panelists: Jack Dinsdale, Utah Office of the Legislative Auditor General

Derek Hippler, Wisconsin Legislative Audit Bureau

Barton Norfleet, Mississippi PEER Committee

Concurrent Session 2—Housekeeping: Cleaning Data
Madison Concourse, 2nd Floor, Capitol Ballroom B

Data analysis techniques learned in public policy schools may not fully anticipate the legislative evaluator’s burden to clean data. Using experiences derived from actual evaluation work, this session will explore strategies for cleaning data effectively.

Moderator: Wayne Kidd, Utah Office of the Legislative Auditor General

Panelists: Kirby Arinder, Mississippi PEER Committee

Dan Kleinmaier, Wisconsin Legislative Audit Bureau

Ryan Moltz, Minnesota Office of the Legislative Auditor

Concurrent Session 3—Figure It Out! Data Visualization and Report Design
Madison Concourse, 2nd Floor, Madison Ballroom

Evaluators LOVE numbers…and hence love to show data in tabular form. In this session, we throw open the doors of innovation to study ways in which a figure or graphical presentation of data may better serve the particular writing purpose. If a picture is worth a thousand words, then there are more than one million words to explain how your use of figures can improve the readability of your report. The best evaluation finding is lost on a legislative audience if packaged ineffectively. This session explores innovations and best practices in audit report design that have maximized the accessibility of information to the non-technical audience.

Moderator: Kevin Ryan, South Carolina Legislative Audit Council

Panelists: Katherine Theisen, Minnesota Office of the Legislative Auditor

Emily Schroeder, Washington Joint Legislative Audit and Review Committee

Maria Toniolo, Wisconsin Legislative Audit Bureau

Concurrent Session 4—Evaluating Human Service Programs: The Data-Driven Evaluatoar
Madison Concourse, 2nd Floor, Capitol Ballroom B

Medical assistance, behavioral health, child welfare, early childhood, workers compensation and vocational rehabilitation are examples of the human services and workforce development programs legislative evaluators are called to evaluate. In this session, we look at how data-rich programs such as these provide evaluation opportunities, pose particular challenges for the evaluator and gather strategies for evaluating these types of programs as well.

Moderator: Melinda Hamilton, Michigan Office of the Auditor General

Panelists: Laura Brauer, Wisconsin Legislative Audit Bureau

Sarah Dinces, New Mexico Legislative Finance Committee

Karen Leblanc, Office of the Louisiana Legislative Auditor

Plenary Session 2—Impact Award Information Fair 

Madison Concourse, 2nd Floor, Wisconsin Ballroom 

In this session, winners of 2017 NLPES Certificates of Impact will be available to discuss their winning reports. Please stop by each of the more than 20 tables to view and learn about the work of our colleagues in other states.

Dinner on Your Own

After dinner (approximately 7:30-9 p.m.), you may wish to join your colleagues lakeside at the Memorial Union Terrace to continue networking.

Concurrent Session 5—Constructing Effective Findings

Madison Concourse, 1st Floor, Assembly

This session unpacks the key elements necessary to create and communicate a compelling analytical conclusion in evaluative work. In addition, we consider how evaluators know when each of these elements have been adequately addressed and how to best communicate findings to a legislative audience.

Moderator: Melissa Simpson, Arkansas Division of Legislative Audit

Panelists: Beth Ashcroft, Maine Office of Program Evaluation and Government Accountability

Paul Stuiber, Wisconsin Legislative Audit Bureau

Matt Taylor, Georgia Department of Audits and Accounts

Concurrent Session 6—Training and Retaining Evaluators
Madison Concourse, 1st Floor, Senate AB

Many offices have created in-house training programs to develop the competencies of evaluation staff. In this session, we will explore in-house training programs, what works, what does not and why.

Moderator and Panelist: Dean Swenson, Wisconsin Legislative Audit Bureau

Panelists: Matthias Boon Utah Office of the Legislative Auditor General

Greg Fugate, Colorado Office of the State Auditor

Plenary Session 3—Communicate by Design
Madison Concourse, 2nd Floor, Capitol Ballroom A

Commonly, people struggle with communications in the workplace, especially addressing unmet expectations. Communicating by Design was developed to address this struggle. This session will help us understand our natural human tendencies to default to disrespectful communication under pressure. This session will introduce tools to help us communicate more effectively.

Speaker: Aleta Norris, Partner and Co-Founder, Living as a Leader, Brookfield, Wisc.

Lunch and Plenary 4—Considering Cost-Benefit Analyses

Madison Concourse, 2nd Floor, Wisconsin Ballroom

Legislative evaluators have many methodological tools available to approach a public policy issue and evaluate government performance. In this session, Professor Weimer offers his insights about the importance of keeping a broad view in mind when selecting which tool is appropriate for which analytical challenge.

Speaker: Professor David Weimer, Robert M. La Follette School of Public Affairs, University of Wisconsin-Madison

Concurrent Session 7—Grooming Emerging Leaders
Madison Concourse, 1st Floor, Senate AB

The changing world of work will continue to press organizations to have systems in place to intentionally develop emerging talent. In this session, we will review best practices to enhance your ability to grow your future leaders—a key strategy for attracting and retaining great employees. We will learn: Who are “emerging leaders” and why is it important to development them intentionally? What obstacles and common pitfalls prevent effective development of emerging leaders? What are the five learning strategies for grooming emerging leaders? What is a framework for drafting a development plan for particular emerging leaders in your organization?

Speaker: Aleta Norris, Co-Founder, Living as a Leader, Brookfield, Wisc.

Concurrent Session 8—Blueprint for Success: Developing Core Competencies
Madison Concourse, 1st Floor, Assembly
By focusing on core competencies such as critical thinking skills, this session will focus on ways for those with five or fewer years of experience to advance their careers in evaluation.

Speaker: Scott Frank, Kansas Legislative Division of Post Audit

Concurrent Session 9—Performance Measurement Information: A Behavioral Perspective

Madison Concourse, 1st Floor, Senate AB

Auditors, evaluators, public managers, elected officials and the public are awash with more performance data than ever before. Judicious use of performance information is hoped to provide a basis for improving government services. But how much do we really know about how individuals process performance data? This session explores the cognitive biases that limit our ability to use data rationally and offers suggestions to overcome those biases.

Speaker: Professor Don Moynihan, Director, Robert M. La Follette School of Public Affairs, University of Wisconsin-Madison

Concurrent Session 10—Getting It Right: The Quality Assurance Imperative
Madison Concourse, 1st Floor, Assembly

Good evaluation reports exemplify accuracy. In this session, we study strategies for ensuring the accuracy of reports, including double-checking work, team leader reviews and other quality assurance steps.

Moderator: Pat Berger, Pennsylvania Legislative Budget and Finance Committee

Panelists: Carleen Armstrong, Colorado Office of the State Auditor

Meghan Flanders, Kansas Legislative Division of Post Audit

Valerie Whitener, Washington Joint Legislative Audit and Review Committee

Plenary Session 5—Evaluating Our Ethics
Madison Concourse, 2nd Floor, Wisconsin Ballroom

There are ethical requirements most are expected to meet, and then there are the ethical demands of being an auditor/evaluator. The space between the floor of ethical behavior and the expectations of the public for evaluators is important for us to recognize if we are to succeed. In this session our presenters will explore both Big “E” and little “e” ethics in the context of legislative evaluation and, particularly, with respect to our data responsibilities

Panelists: Mark Quiner, NCSL Center for Ethics in Government

Anne Sappenfield, Wisconsin Legislative Audit Bureau

Concurrent Session 11—Recruiting and Onboarding Evaluators
Madison Concourse, 1st Floor, Assembly

Recent data suggests a decline in the number of public policy graduates nationwide. In addition, evaluation offices indicate it takes more than one year to onboard new staff to a level that offers the organization a return on investment. In this session, we examine strategies for recruiting new staff, building effective relationships with the public policy institutions that train future evaluators and consider ways to onboard staff quickly in order to equip them to contribute more substantially to the work of the organization.

Moderator: Emily Johnson, Texas Sunset Advisory Commission

Panelists: Professor Hilary Shager, Associate Director, Robert M. La Follette School of Public

     Affairs, University of Wisconsin-Madison

Suzanna Pratt, Washington Joint Legislative Audit and Review Committee

Kristen Rottinghaus, Kansas Legislative Division of Post Audit

Kyle Swalls, Wisconsin Legislative Audit Bureau

Concurrent Session 12—Using Tools for Data Analytics

Madison Concourse, 1st Floor, Senate AB

Contemporary program evaluation relies increasingly on the evaluator’s skill in understanding and using data. In this session, we will explore tips and tricks for using Excel, ACL, SQL, R, and ArcGIS/ArcMap successfully in our work.

Moderator: Jon Courtney, New Mexico Legislative Finance Committee

Panelists: Andy Brienzo, Kansas Legislative Division of Post Audit

Zane Kelly, Washington Joint Legislative Audit and Review Committee

Chris Magee, Louisiana Legislative Auditor

Lunch and Plenary 6—The Conversation Catalyst
Madison Concourse, 2nd Floor, Wisconsin Ballroom

The Urban Dictionary defines a conversation catalyst as “the introduction of a new thread in a conversation, made to keep an otherwise flagging conversation ongoing.” In this roundtable discussion, the moderator will introduce several “new threads” to glean the perspectives of others in the room. If you don’t see your capital idea represented elsewhere in the 2017 seminar agenda, this is an excellent opportunity to solicit input from fellow attendees.

Moderator:  Joe Chrisman, Wisconsin Legislative Audit Bureau

Concurrent Session 13—NLPES Award-Winning Session
Madison Concourse, 1st Floor, Senate AB

Each year, NLPES recognizes the hard work done by legislative performance audit and program evaluation offices by offering awards in four categories for which an office may submit a nomination or application: Outstanding Achievement Award, Excellence in Evaluation Award, Excellence in Research Methods Award, and Certificates of Impact.  In this session, the winner of the 2017 Excellence in Research Methods Award (Utah Office of the Legislative Auditor General) describes its winning work.  In addition, members of the NLPES Executive Committee, Awards Subcommittee, discuss the overall awards program and the opportunities it presents for the involvement and recognition of member organizations.

Moderator: Melinda Hamilton, Michigan Office of the Auditor General

Panelists: Jon Courtney, New Mexico Legislative Finance Committee

Emily Johnson, Texas Sunset Advisory Commission

Marcia Lindsay, South Carolina Legislative Audit Council

Darin Underwood, Utah Office of the Legislative Auditor General

Concurrent Session 14—Writing ABCs
Madison Concourse, 1st Floor, Assembly

Good evaluation reports demonstrate three principles of writing: accuracy, brevity and clarity (ABCs). This session will unpack each principle to understand how best to translate technical information to a non-technical audience.

Moderator: Joel Alter, Minnesota Office of the Legislative Auditor

Panelists: Yvonne Benn, Michigan Office of the Auditor General

Kiernan McGorty, North Carolina Program Evaluation Division

Robert Romig, Texas Sunset Advisory Commission

Concurrent Session 15—Evaluating Transportation Programs: Avoiding the Highway to the Danger Zone
Madison Concourse, 1st Floor, Senate AB

Many states are confronted with aging transportation infrastructure, including bridges, highways and mass transit alternatives. In this session, we consider approaches used to evaluate transportation programs for efficiency and effectiveness while avoiding the highway to danger.

Moderator: Shunti Taylor, Georgia Department of Audits and Accounts

Panelists: Joe Murray, Montana Legislative Audit Division

Noah Natzke, Wisconsin Legislative Audit Bureau

Jennifer Sebren, Mississippi PEER Committee

Concurrent Session 16—Accepting Feedback
Madison Concourse, 1st Floor, Assembly

We train our team leaders and managers on ways to provide meaningful and constructive feedback to staff, but how much time is spent to teach all staff the skills necessary to accept feedback well? In this session, we offer a framework for accepting feedback that works for any individual in any organization.

Speaker: Joe Chrisman, Wisconsin Legislative Audit Bureau

Plenary Session 7—Reflecting Capital Ideas for Legislative Evaluation
Madison Concourse, 2nd Floor, Wisconsin Ballroom

The field of legislative program evaluation is a challenging and rewarding profession. In this session, three leaders describe their own emerging “capital ideas” about the profession, its challenges and its future.

Panelists: Joe Chrisman, Wisconsin Legislative Audit Bureau

Scott Frank, Kansas Legislative Division of Post Audit