2024 NLPES Professional Development Seminar

Monday, Oct. 7 - Registration at the Hotel Monteleone

4-6 p.m. Queen Anne Mezzanine

Registration and Information Desk

Beat the rush and check-in early. Don't forget your badge and meeting packet!

4-5:30 p.m.

Queen Anne A

NLPES Executive Committee Meeting

Presiding, Jennifer Sebren, deputy director, Mississippi PEER Committee

Tuesday, Oct. 8 - All Sessions at the Hotel Monteleone

8 a.m.-4 p.m.

Queen Anne Mezzanine

Registration and Information Desk

Check-in, get your badge and meeting packet before the sessions start!

8:30-9:30 a.m.

Queen Anne/Parlor

Plenary Session: Welcome to NOLA and Legislative Perspectives

This plenary session will kick off the PDS with presentations from Louisiana's Legislative Auditor (Mike Waguespack) and area legislators who will speak on the value of audits and evaluations

Moderators:Mike Waguespack, legislative auditor, Louisiana Legislative Auditor Karen Leblanc, director, Performance Audit Services, Louisiana Legislative Auditor

Speakers: Members of Louisiana Audit Advisory Council

Plenary

9:30-9:45 a.m.

Queen Anne Foyer

Beignets and Café Au Lait

Grab a morning snack and enjoy some southern hospitality with beignets and café au lait.

Snack

CONCURRENT SESSIONS

9:45-10:45 a.m.

Queen Anne/Parlor

Strategic Audit Planning

A successful audit begins with a well-thought-out plan. But how do we get there? How do we know if we are evaluating the right areas the right way? States will share how their offices choose audit topics, their processes to plan effective and efficient audits, and the lessons they have learned along the way.

Moderator: Kristen Jacobs, senior performance auditor, Louisiana Legislative Auditor

Speakers: McKenzie Cantlon, lead auditor, Utah Office of the Legislative Auditor General Ashley Brecheen, senior performance auditor, Louisiana Legislative Auditor Nehemiah Chinavare, assistant principal performance auditor, Wisconsin Legislative Audit Bureau

Track

Junior Evaluators

9:45-10:45 a.m.

Royal Salon

Evaluating our Work: Measuring Impact

Panelists will discuss ways they assess the impact of their work, including follow-up processes, calculating financial benefits, and other qualitative and quantitative measures that demonstrate the value of their work.

Moderator:Krista Baker-Hernandez, performance audit manager, Louisiana Legislative Auditor

Speakers:Nina Frant, performance audit quality assurance director, Colorado Office of the State Auditor Marcus Morgan, director, Alabama Commission on the Evaluation of Services Becky Beachy, audit manager, Texas State Auditor's Office

Track

Senior Evaluators

9:45-10:45 a.m.

Queen Anne B

Evaluating Elections

This session will focus on three recent audits of elections. Utah's review of the 2022 election found no evidence systematic problems, widespread errors, or significant fraud. Wisconsin's evaluation focused on a broad range of elections administration topics, Louisiana's audit addressed controls to ensure the accuracy of voter registration data, and the state's implementation of best practices from the U.S. Election Assistance Commission.

Moderator:Mohri Exline, senior auditor, Kansas Legislative Division of Post Audit

Speakers: Julie Floyd, senior performance auditor, Louisiana Legislative Auditor Leah Blevins, audit manager, Utah Office of the Legislative Auditor General Ross Ryan, principal performance evaluator, Wisconsin Legislative Audit Bureau

Track

What We Evaluate

CONCURRENT SESSIONS

11 a.m.-Noon Queen Anne/Parlor

Gathering Evidence/Developing Conclusions and Findings

Collecting and determining how much evidence is needed is vital to successfully developing well supported conclusions and findings. This panel will be an interactive experience where participants will be provided different evaluation scenarios and asked to brainstorm ways to collect sufficient evidence and then how best to develop findings and conclusions based on that evidence. To include different perspectives, the panelists represent states that both follow and do not follow Government Auditing Standards (known as the "Yellow Book").

Moderator: Gina Brown, performance audit manager, Louisiana Legislative Auditor

Speaker: Rachel Kelly, senior performance auditor, Louisiana Legislative Auditor

Patrick Dean, assistant director, Alabama Commission on the Evaluation of Services

Track

Junior Evaluators

11 a.m.-Noon Queen Anne B

Beyond Audits and Evaluations: How We Help in Other Ways

Panelists from Washington, Maryland, and Utah will share examples of assignments and initiatives that support efficient and effective government functioning beyond audits and evaluations. Short presentations will be given on 1) The Center for Government Innovation in Washington; 2) Evaluating in real-time: The COVID vaccination experience; and 3) Utah's high-risk list and best practices handbooks.

Moderator: Melanie Stidham, research analyst, Washington Joint Legislative Audit and Review Committee

Speakers: Darin R. "DRU" Underwood, deputy auditor general, Utah Office of the Legislative Auditor General Michael Powell, director, Maryland Office of Program Evaluation and Government Accountability Jolene Stanislowski, performance auditor, Washington State Auditor's Office

Track

Senior Evaluators

11 a.m.-Noon Royal Salon

Evaluating Vulnerable Populations: Methods and Ethics

Evaluating programs for vulnerable populations-such as those with severe mental illness, incarcerated people, and nursing home residents-can be challenging, especially when performance data are lacking. Evaluators must select methods that are defensible, practical and ethically sound. In this session, panelists will share insights from evaluations of state psychiatric hospitals, child protection services and correctional programs, highlighting key lessons learned.

Moderator: Jason Juffras, senior analyst, Office of the District of Columbia Auditor Speakers: Jodie Debetaz, senior performance auditor, Louisiana Legislative Auditor

Drew Dickinson, chief legislative analyst, Virginia Joint Legislative Audit and Review Commission David Kirchner, evaluation coordinator, Minnesota Office of the Legislative Auditor

Track What We Evaluate

12:15-1:15 p.m. Queen Anne/Parlor

Lunch Plenary: NLPES Awards Ceremony

Grab a po'boy and sweet tea and let's recognize and celebrate the winners of this year's Outstanding Achievement Award, Excellence in Evaluation Award, Excellence in Research Methods Awards and Certificates of Impact (see awards list on the app for winners).

Presiding:Jennifer Sebren, NLPES chair 2024-2025, Mississippi Jeanine Brown, NLPES Awards Committee chair, 2023-2024 Speaker:John Snyder, NCSL Staff Chair, Kentucky

Plenary

CONCURRENT SESSIONS

1:30-2:30 p.m. Queen Anne/Parlor

Designing Meaningful Surveys-Roundtable

Surveys are an efficient means to gather information and collect opinions simultaneously from a large number of people, but their effectiveness depends on the quality of their design. Because evaluators are using surveys much more frequently, it is important to consider how well we are designing them. A well-constructed survey not only motivates a high response rate, but helps to ensure the responses themselves are valid, reliable and can add value to our evaluations

Moderator:Amanda G. Sanford, senior performance auditor, Louisiana Legislative Auditor

Speakers: Savana Griffin, policy lead, Alabama Commission on the Evaluation of Services Sam Rebenstorf, assistant principal performance evaluator, Wisconsin Legislative Audit Bureau Lauren Bailey, senior evaluator, Idaho Office of Performance Evaluations

Track

Junior Evaluators

1:30-2:30 p.m.

Royal Salon

IT Related Issues and Reviews

This session will focus on a broad array of IT evaluation topics, including auditing practices in cybersecurity, IT project oversight, and assessing data reliability from existing IT systems. It will cover Utah's Office of Legislative Auditor General's 2023 audit of the cybersecurity posture of government agencies, highlight findings from the Arizona Auditor General's performance audit of the state government IT project oversight, and introduce tools for assessing data reliability per Yellow Book standards.

Moderator: Micaela Fischer, deputy director for program evaluation, New Mexico Legislative Finance Committee

Speakers: Kade Minchey, auditor general, Utah Office of the Legislative Auditor General John Clinch, senior performance audit manager, New Hampshire Miguel Caro Mateus, performance audit manager, Arizona Auditor General

Track

Senior Evaluators

1:30-2:30 p.m.

Queen Anne B

Workforce Development Evaluations

Workforce development encompasses a wide range of programs and services to help employees improve their job skills, career opportunities and earnings, and to help employers fill workforce shortages and avoid layoffs. Federal and state government investments in workforce development make it imperative that states and agencies implement effective programs with measurable outcomes, including those focused on at-risk populations. Presenters from three states will share challenges, findings and recommendations from their respective workforce development program evaluations.

Moderator: Jennifer Sebren, deputy director, Mississippi PEER Committee

Speakers:Tori Hall, principal evaluator, Maryland Office of Program Evaluation and Government Accountability Sarah Dinces, principal program evaluator, New Mexico Legislative Finance Committee Chris Magee, data analytics manager, Louisiana Legislative Auditor

Track

What We Evaluate

CONCURRENT SESSIONS

2:45-3:45 p.m. Queen Anne/Parlor

From Draft to Impact: Transforming Reports with Purpose

Discover strategies to refine reports that engage different audiences, emphasize core ideas and communicate with clarity and precision.

Moderator:Patrick Dean, assistant director, Alabama Commission on the Evaluation of Services

Speakers: Macie Smith, Kansas Legislative Division of Post Audit Kristen Jacobs, senior performance auditor, Louisiana Legislative Auditor Olha Bilobran, Washington State Auditor's Office

Track

Junior Evaluators

2:45-3:45 p.m.

Queen Anne B

Innovative Research Methods - Winners of 2024 Methodology Awards

Listen to evaluators from Alabama and Kansas talk about their award-winning methodologies and get inspiration for how you can adapt them to your state's needs. This year's Excellence in Research Methods Award Winners will talk about how their innovative methods allowed them to leverage data from across state government to answer questions specifically related to education.

Reports discussed:

A Follow-Up Evaluation of the Teacher Workforce , Alabama Commission on the Evaluation of Services

Estimating the Cost of K-12 Education , Kansas Legislative Division of Post Audit

Moderator: Courtney Stevenson, senior research methodologist, Louisiana Legislative Auditor Speakers: Marcus Morgan, director, Alabama Commission on the Evaluation of Services Cade Graber, senior auditor, Kansas Legislative Division of Post Audit Savana Griffin, policy lead, Alabama Commission on the Evaluation of Services

Track

Senior Evaluators

2:45-3:45 p.m.

Royal Salon

Auditing Organizational Culture

The culture of an organization impacts efficiency, effectiveness and the ability of an entity, such as a state agency, to fulfill its mission. In this session, you will hear about recent projects that evaluated organization culture of agencies in Louisiana and Utah including the Department of Children and Family services, state prisons, and smaller agencies. We will discuss strategies, such as surveys and interviews, for obtaining evidence to evaluate organizational culture and have time for Q&A.

Moderator: Julie Floyd, senior performance auditor, Louisiana Legislative Auditor Speakers: Brian Dean, deputy auditor general, Utah Office of the Legislative Auditor General

Emily Dixon, performance audit manager, Louisiana Legislative Auditor

Track

What We Evaluate

4:30-6 p.m.

Royal Ballroom

Certificate of Impact Award: Poster Session and Happy Hour

Learn about this year's Certificate of Impact winning reports. Grab a refreshment and nosh while you chat and check out the posters!

Plenary

6 p.m.

Explore the French Quarter and Grab Dinner on Your Own

Wednesday, Oct. 9 - All Sessions at the Hotel Monteleone

8 a.m.-4 p.m. Queen Anne Mezzanine

Registration and Information Desk

Check-in, get your badge and meeting packet before the sessions start!

7-8 a.m.

Breakfast on Your Own

8-9:15 a.m. Queen Anne/Parlor

Staying Relevant in a Divided Society

As the country faces increasing polarization, auditors and evaluators can serve a critical role for state legislatures operating in a divided environment. The former legislative auditor for Washington state will reflect on his career and provide suggestions for how auditors can stay relevant and assist legislatures in these challenging times.

Speaker:Keenan Konopaski, Washington state legislative auditor (retired) and 2024 NLPES Outstanding Achievement Award Winner

Plenary

CONCURRENT SESSIONS

9:30-10:30 a.m.

Queen Anne/Parlor

Conducting Effective Interviews

Our goals for this session are to explain what an effective interview is, take you through the process of preparing for the interview, conducting the interview and documenting the interview. Plus, tips and tricks to help you become a better communicator. Finally, we will tie together all these facets in hopes they can be used to create and document your audit findings more effectively.

Moderator: Lauren Whatley, senior performance auditor, Louisiana Legislative Auditor

Speakers: John Holland, assistant principal performance evaluator, Wisconsin Legislative Audit Bureau John Harrington, management and program analyst manager, Montana Legislative Audit Division Stephanie Winzeler, performance audit supervisor, Colorado Office of the State Auditor

Track

Junior Evaluators

9:30-10:30 a.m.

Royal Salon

Developing a Training Curriculum: Roundtable

This roundtable will include a discussion on how states develop training that meets the needs of all their staff. Each panelist will discuss the training in their offices, advantages and challenges in assessing and planning training, and innovative or unique ideas around training. Moderator: Marcia A. Lindsay, deputy director, South Carolina Legislative Audit Committee

Speakers: Jenny Page, director of strategic operations, Colorado Office of the State Auditor Jodi Munson Rodriguez, deputy legislative auditor, Minnesota Office of the Legislative Auditor Krista Baker-Hernandez, performance audit manager, Louisiana Legislative Auditor

Track

Senior Evaluators

9:30-10:30 a.m.

Queen Anne B

Evaluating Medicaid

Each state's Medicaid program involves many levels of administration and impacts different groups of stakeholders. Panelists will discuss different approaches they have taken to auditing their states' oversight of Medicaid, with areas of focus ranging from beneficiary access to services, monitoring providers and managed care organizations, and holding the state accountable for fulfilling its overall responsibilities.

Moderator:Lauren Pendas, senior performance auditor, Louisiana Legislative Auditor

Speakers: Chris Magee, data analytics manager, Louisiana Legislative Auditor

Ryan Langrill, interim director, Idaho Office of Performance Evaluations

Brent McDougall, performance audit senior data analyst, Louisiana Legislative Auditor

Track

What We Evaluate

CONCURRENT SESSIONS

10:45- 11:45 a.m. Queen Anne/Parlor

Making Fieldwork Work (and Even Fun!)

Crossing the fieldwork finish line into reporting can sometimes seem like a daunting task, especially when walking the fine line between completing work on time and producing a quality, impactful end product. Participants will leave this panel with tips on how to overcome common challenges faced during fieldwork based on actual evaluations.

Moderator: Gina Brown, performance audit manager, Louisiana Legislative Auditor

Speakers:Mohri Exline, senior auditor, Kansas Legislative Division of Post Audit Misty Perry, senior performance auditor, Louisiana Legislative Auditor Mackenzie Moss, senior evaluator, Idaho Office of Performance Evaluations

Track

Junior Evaluators

10:45- 11:45 a.m.

Queen Anne B

Working with Specialists: When, Why and How

Panelists will share examples of working with external specialists who offer special skills and expertise. The discussion will include the various roles these experts play in the audit and evaluation process and how to manage risks involved in engaging outsiders.

Moderator: Michael Powell, director, Maryland Office of Program Evaluation and Government Accountability

Speakers:Tina Chamberlin, associate management and program analyst, Montana Legislative Audit Division Lonnie Edgar, deputy director, Mississippi PEER Committee Nathan Clubb, senior analyst, Georgia Department of Audits and Accounts

Track

Senior Evaluators

10:45- 11:45 a.m.

Royal Salon

Capital/Infrastructure Projects

Capital and infrastructure projects, which can last for years or even decades, represent a substantial commitment of public resources and can face intense political and regulatory scrutiny. Evaluating these projects poses unique challenges due to their size, technical and political complexity, and long timelines. Panel members with diverse evaluation experiences including transportation, port, school and prison projects, will offer insights on overcoming obstacles and effectively evaluating capital and infrastructure projects

Moderator:Matt Fahrenbruch, senior performance auditor, Kansas Legislative Division of Post Audit

Speakers: Irina Hampton, senior performance auditor, Louisiana Legislative Auditor

Casey Petti, principal evaluator, Idaho Office of Performance Evaluations

David Kirchner, evaluation coordinator, Minnesota Office of the Legislative Auditor

Track

What We Evaluate

Noon-1:15 p.m. Queen Anne/Parlor

Lunch with Patrick Goldsmith: Lessons Learned from the Other Side

Get a plate of N'Awlins grub and sit down for a talk with Patrick Goldsmith. He'll give his insights about communicating with the executive branch and what he came to appreciate about the legislature after crossing over... to the other side.

Speaker:Patrick Goldsmith, deputy commissioner, Louisiana Division of Administration, former director of the Louisiana House Fiscal Division and former director of Performance Audit at the Louisiana Legislative Auditor.

* Tribute to Greg Fugate will also take place. Presented by Nina Frant with the Colorado Office of the State Auditor

Plenary

CONCURRENT SESSIONS

1:30-2:30 p.m.

Queen Anne/Parlor

How to Succeed as a Junior Evaluator-Roundtable

This roundtable discussion on the qualities of successful Junior Evaluators, will consider the unique expectations and challenges faced by new evaluators and opportunities to support their growth and development.

Moderator:Emily Braun, staff performance auditor, Louisiana Legislative Auditor

Speakers:Dean Swenson, deputy state auditor for performance evaluation, Wisconsin Legislative Audit Bureau Drew Johnson, analyst, Mississippi PEER Committee Jodi Munson Rodriguez, deputy legislative auditor, Minnesota Office of the Legislative Auditor

Track

Junior Evaluators

1:30-2:30 p.m. Queen Anne B

Enhancing Our Results - Use of Different Report Formats

Looking for ways to present your results that will affect more positive change in government? Unlock the potential to enhance your results by discovering alternative approaches and products that member offices use to report audit work. This session will discuss innovative approaches to expand beyond the traditional hard copy report and potentially reach a wider audience.

Moderator:Jeanine Brown, senior management analyst, Georgia Department of Audits and Accounts

Speakers:Janna Abraham, management analyst III, Georgia Department of Audits and Accounts Sarah Carlson, senior management and program analyst, Montana Legislative Audit Division Alison Butts, senior performance auditor, Louisiana Legislative Auditor

Track

Senior Evaluators

1:30-2:30 p.m.

Royal Salon

Evaluating Education

In this session, panelists will present different methods they use to evaluate education in their states. They will use their experience and expertise to provide ideas for other states about what data is useful, what's actually important and what's not, and how this information can be shared.

Moderator: Courtney Stevenson, senior research methodologist, Louisiana Legislative Auditor

Speakers:Jennifer Sebren, deputy director, Mississippi PEER Committee Darin "DRU" Underwood, deputy auditor general, Utah Office of the Legislative Auditor General

Marcus Morgan, director, Alabama Commission on the Evaluation of Services

Track

What We Evaluate

2:45-4 p.m.

Queen Anne/Parlor

Managing Partisan Culture & Conflict in the Legislative Evaluation Lifecycle

This session will explore practical ways to build and stabilize a supportive organizational culture amidst the many partisan pressures, conflicts, and difficult conversations regularly occurring for professional legislative program evaluators.

Speaker:Andrew Marin, PhD, senior advisor, The Clearing Inc.

Plenary

5:30-6:30 p.m.

Royal Ballroom

Networking Happy Hour with NALFO

Let the Good Times Roll! Gather for a networking happy hour in the Royal Ballroom with our colleagues from the National Association of Legislative Fiscal Offices (NALFO)!

Social Event

Thursday, Oct. 10 - All Sessions at the Hotel Monteleone

8 a.m.-Noon

Information Desk

8-9:15 a.m. Queen Anne/Parlor

Evaluating for Impact: Rebuilding Public Trust

In an era when polling tells us that public trust in government is hovering near historic lows, legislative oversight is more important than ever. Join Jim Townsend, director of the Carl Levin Center for Oversight and Democracy, for a discussion of the important role performance evaluation plays in legislative oversight and the potential it has to demonstrate program effectiveness, improve governance and rebuild public trust in our institutions.

Speaker: James Townsend, director, Levin Center for Oversight and Democracy

Plenary

CONCURRENT SESSIONS

9:30-10:30 a.m.

Queen Anne/Parlor

Creative Data Analytics

In this session, experts from various states will share insights on their innovative use of data analytics on recent audits. Kansas used machine learning applications and will share how it used a neural network to evaluate fraud in unemployment insurance claims. Louisiana has used both internal and external datasets to detect inefficiencies, fraud, mismanagement and errors. Colorado used a variety of techniques on an audit of its Enterprise Zones Tax Credit.

Moderator: Mukta Pathak, senior performance auditor, Louisiana Legislative Auditor

Speakers:Brent McDougal, performance audit senior data analyst, Louisiana Legislative Auditor Trey Standley, performance audit and tax expenditure evaluation manager, Colorado Office of the State Auditor

Cade Graber, senior auditor, Kansas Legislative Division of Post Audit

Track

Junior Evaluators

9:30-10:30 a.m.

Royal Salon

Enhancing Morale and Engagement-Roundtable

What strategies can offices use to create a workplace where employees are engaged, productive, and fulfilled? Join our roundtable discussion and explore how states have made their offices of different sizes great places to work. We will discuss creating a culture of accountability that normalizes feedback, communicating effectively, generational differences in motivation, encouraging a work-life balance, and, of course, what can make offices fun! Moderator: Kristen Jacobs, seniorperformance auditor, Louisiana Legislative Auditor

Speakers:Marcus Morgan, director, Alabama Commission on the Evaluation of Services Becky Beachy, audit manager, Texas State Auditor's Office

Track

Senior Evaluators

9:30-10:30 a.m.

Queen Anne B

Auditing Natural Resources and Environmental Issues

Effective management and oversight, as well as sustainable use, of natural resources are crucial for protecting both the public and the environment in our states. Panelists from Colorado, Kansas, and Utah will discuss challenges they encountered when evaluating water management and gas pipeline safety in their states, how they overcome these challenges, and why these types of evaluations are important.

Moderator: Irina Hampton, senior performance auditor, Louisiana Legislative Auditor

Speakers:Jenny Page, director of strategic operations, Colorado Office of the State Auditor Madison Hoover, performance audit supervisor, Utah Office of the Legislative Auditor General

Sarah Middendorf, auditor, Kansas Legislative Division of Post Audit

Track

What We Evaluate

10:45-11:45 a.m. Queen Anne/Parlor

Plenary Session: Teambuilding Activity Lead by Louisiana Host State Staff

During this session, we will work together to create something that each attendee can take home as a reminder of his/her time in New Orleans. The activity mimics several aspects of team project work while highlighting how everyone's piece of the puzzle is important in creating the big picture.

* This team-building exercise will use the canvas tote bag that everyone received at check-in. Please do not pack it away or discard it if you plan on participating in this activity.

Plenary

Noon

SEMINAR CONCLUDES