NLPES Site Map

NLPES Question of the Month

August/September 2001

Based on your experience in legislative program evaluation so far, what are the most significant attractions and the most difficult challenges of your daily work? (Note: This question was directed to staff who had been working in a legislative program evaluation office for 5 years or less.)

From: Rodney Ferguson, Kansas

Because today marks my 6-month anniversary with the Legislative Division of Post Audit for the State of Kansas, I am extremely well qualified to provide a response to this month’s question.

By far the most stimulating attraction to the work is the constant opportunity for discovery. As a new audit begins to take form in the initial meetings, it’s very enjoyable to ‘free range’ in the pursuit of basic background information. Also when the audit guide is developed listing individual specific assignments, it is almost like having your very own officially sanctioned ‘scavenger hunt’, coupled with the fact that you’re getting paid!

Challenging is the immutable requirement for extreme attention to small detail, which is vital to the credibility of the finished product.

Having fun, being challenged, and getting paid for it, is the short answer.

It would be interesting to ask again in 234 weeks.

From: Wisconsin Legislative Audit Bureau staff

At the Wisconsin Legislative Audit Bureau, we polled several program analysts with 3 years or less work experience (all of which are in non-supervisory positions). These analysts identified 4 attractions and 4 challenges to our daily work.

Attractions identified include:

Variety of audits – because we evaluate a wide range of government programs, there is always something new to learn, and we avoid being pigeonholed into evaluating one subject area.
Development of expertise – while we may not focus exclusively on one or two policy areas, our audits are thorough and in-depth. This allows us to develop an expertise in that subject area, as well as enhance our research and analytical skills.
The underlying mission of our daily work – the ability to improve government is another attraction of working as a program evaluator. A reward unique to government service is earned when the recommendations offered in our work are debated and implemented by the Legislature and the Department.
A generally consistent and stable work environment – there is neither a great deal of day-to-day schedule fluctuations, nor worries of pending staff reductions, which are commonly associated with other employers, especially in the private sector. While longer hours may occasionally be required, we are able to predictably structure our days and lives outside of the office.

While working as a program evaluator has many appealing attributes, it is not without some challenges.

Too much information – an endless amount of information proves to be a daily obstacle for program evaluators. In the beginning of an audit, the learning curve is intimidating. Added with the advent of the Internet and other electronic data sources, it is also hard to know when to stop collecting information as well as how to determine what information will prove to be the most reliable and useful.
Establishing criteria – the criteria for evaluating a program are rarely clear-cut. Once the criteria are established, reliable data may not be readily available for evaluating based on the chosen criteria.
Taking complex and daunting government programs and writing about them in a clear, concise, and easily understood manner – effectively communicating our findings is a daily challenge.
Sacrifices of working in the public sector -- while we chose to work in the public sector for the previously stated reasons among others, it is likely that we could be earning higher salaries had we opted for employment in the private sector. This is a reality we are reminded of with each student loan payment.

From: Andy Slain, Nebraska

I have worked in the Nebraska Legislative Program Evaluation Unit for a little over two-and-a-half years as both an analyst and legal counsel. The aspect of my daily work that I find most challenging occurs in the early stages of an evaluation. I think it is a challenge to determine the proper scope of an evaluation and then find ways to answer the evaluation questions you've identified. It is fairly typical in our unit to pose a couple of descriptive questions (e.g., What are the goals of the program? Are they complying with their statutes?) and then one or two evaluation questions (e.g., Has the program met its goals? Is there still a need for the program?) as part of the scope. The evaluation questions are invariably difficult to quantify and that never fails to give me pause. We have always had good success gathering qualitative information from agency staff and shareholders, and then applying our own analysis, but for me it lacks the satisfaction of a more definitive, quantitative analysis. This is the nature of the work and we all understand that, but I still feel that thinking about interesting ways to answer difficult questions is the most challenging part of my job. A related challenge is learning to feel comfortable with your analysis when the difficult question forces you to use less-than-ideal measures. That has a lot to do with your confidence that, despite bad circumstances, you have still done the best you could.

For me, there are a number of attractions to the job as well. First, the variation is nice. About the time that I find myself getting deep-down tired or bored with an evaluation, the evaluation ends and it is time to learn something new about a different program. That is appealing. Second, and two years ago I wouldn't have dreamed I’d say this, I love the writing. I have seen so much improvement in my own writing and the unit's reports overall that I get really excited about further progress. I experience a great deal of personal and professional satisfaction when we release a report that we have bled and sweated over. The last thing about the job that I find particularly attractive is my coworkers. Nebraska is a very small shop, the smallest in fact, and there is lots of interaction between myself, the director, and the other analysts. We have our moments of conflict, to be sure, but in the end we play off of each others’ strengths and weaknesses and it is all about the final product. It’s a very nice environment to be in.

So that’s my answer to this month's question. I wish I could say that lack of things to do was a challenge and that the money was an attraction, but overworked and underpaid is the evaluator's way. It didn't take me two-and-a-half years to figure that out, but I'm still here and plugging along so it can't be all bad.

From: Deanna Herring, Utah

I really enjoy the variety in the job. I like learning about all different organizations within state government. Also, one of the biggest reasons I took this job was the flex-time option. One of the most difficult challenges is sometimes being faced with auditees who do not believe I am capable of evaluating their program since I have not worked in their office.

From: Karen Kerr, Mississippi

I’ve been in legislative program evaluation for just over a year and I’d say one of the most significant attractions is variety. I never get bored because I’m always being given different types of assignments. One project is never like another. Also, that leads to the most difficult challenges as well. Sometimes, I’m given assignments involving subject matter that I know nothing about. This is challenging because I have to start from scratch when getting into the project.

From: Greg Spradley, Tennessee

I’ve been doing this stuff for a little over five years, but I did performance audits for about 4 years, 10 months, and became a legislative analyst about eight months ago, so I might qualify.

There are a number of attractions, such as a fairly flexible work schedule and good benefits. Those are almost essential with a family at home. I also enjoy the variety involved with the work, because I could see myself getting really burned out doing the same things over and over again. The fact that I’m not married to my desk and may be talking about water supplies one day and data management the next keeps me interested. Also, while this can be seen as a drawback at times, I get to know a bunch of different people (which may sound corny to some, but I’m generally a people person). A person in this line of work also gets exposed to many different issues that other people in the office are working on, and that’s pretty fun for an information junkie.

Daily challenges include finding enough time in the day to accomplish what I feel I need to, finding new ways to gather the information policymakers need in order to make decisions, making sure that people can use the information we produce, and maintaining an eye for detail while reviewing state rules & regs (a current challenge).

From: Randy Tongier, Kansas

Attractions: Learning something new all the time (getting to do this); working with talented people.

Challenges: Learning something new all the time (having to do this); sometimes looked upon as “the enemy.”