National Program Evaluation Society (NLPES) Executive Committee Meeting Minutes

NLPES Executive Committee Meeting Minutes
Sept. 30, 2012
Atlanta, Georgia

The executive committee met on Sunday, Sept. 30, 2012. Copies of handouts discusssed during the meeting are kept on file with the NLPES Secretary.

Executive Committee Members
Dale Carlson, California
Greg Fugate, Colorado
Rachel Hibbard, Hawai’i
Wayne Kidd, Utah
Lisa Kieffer, Georgia
Marcia Lindsay, South Carolina
Angus Maciver, Montana
Nathalie Molliet-Ribet, Virginia
Carol Ripple, North Carolina
Scott Sager, Wisconsin
Karl Spock, Texas

Brenda Erickson, NCSL liaison

Kathy McGuire, Florida


Karl called the meeting to order.

The committee approved the minutes from the Aug. 7, 2012 meeting in Chicago.

Nomination and election of Vice-Chair and Secretary
Karl first asked for nominations for Vice-Chair. Wayne nominated Lisa Kieffer, and Greg Fugate seconded the nomination. Lisa was approved as the Vice-Chair by the committee.  Second, Karl asked for nominations for Secretary. Lisa nominated Wayne Kidd, and Dale seconded the nomination. Wayne was approved as the Secretary by the committee. 

Overview of LSCC
Karl gave an overview of the Legislative Staff Coordinating Committee (LSCC). We were each given a pamphlet created by a LSCC work group titled NCSL/LSCC Working For You. The pamphlet explains the purpose of the LSCC and provides an organization chart to show how the LSCC operates within NCSL. 

Karl explained that the purpose of the pamphlet is to help staff understand the process and explain how to get more involved with NCSL. 

Karl reviewed the position of the LSCC within NCSL (on page 6 of the pamphlet), and the makeup of the LSCC (on page 8 of the pamphlet). There are 49 people on the LSCC, and the members of the LSCC include the staff section chairs and vice chairs. The LSCC meets four times a year. Karl explained that this is an opportunity to network with other staff sections and discuss general policy issues. 

Karl made us aware that four important items recently discussed at LSCC include:

PDS Locations. Each staff section can coordinate the location of its PDS. In the last couple of years, the staff sections have focused on hub cities to help reduce travel costs, but this may not be emphasized in the future. Kathy mentioned that six staff sections are meeting together this year in Madison, Wisc.
Options for E-Learning. NCSL has many option s for e-learning opportunities and grant money is available to help fund e-learning. Grant money is available for webinars and limited travel.  There has been money left over this past year. Dale asked if the money carried over from year to year. Kathy stated that grant money expires at the end of the year. A discussion among Scott, Brenda, Marcia, and Dale took place about PDS topics for webinars. A live-stream of the PDS sessions would not be a good idea because the quality is not good. However, feedback from PDS participants can be useful to identify sessions that would be well-suited for a webinar.    
LSCC Outreach.There was an orientation for LSCC officers at the Summit, but staff attendance was down this year. LSCC is developing a webinar to provide orientation for incoming officers in the future.
Staff Section Awards.Announcement of staff section awards typically takes place at the Summit. There is a question as to how the announcement of the staff section awards at the Summit will work in the future whether the announcement will be made during a breakfast, lunch, or separate meeting for staff. A decision by the LSCC has not yet been made. Distribution of NLPES’s awards has been moved to the PDS. 


Greg asked about appointments to NCSL standing committees. Brenda stated that staff members of the NCSL standing committees are appointed by heads of their respective legislative agencies and usually remain in that appointment until removed by their legislative agency director. Brenda said that staff officers of NCSL standing committees are one-year appointments.  NCSL standing committees meet three times a year—at the Spring and Fall Forums and the Legislative Summit. 

Update on 2012 PDS
Lisa reviewed the schedule for the fall PDS.  Lisa highlighted a few of the agenda items: 

  • The Welcome Reception would be Monday (October 1) from 5 to 6 p.m. in the Grand Ballroom in the hotel.

  • Carole Buncher, a local consultant, would be providing the training for the data analysis Tuesday morning (October 2).

  • At the Tuesday luncheon, Karl would welcome everyone, Jan Yamane would provide an overview of LSCC, and Lisa would present the awards.

  • The Tuesday evening reception would be held at the Dark Horse Tavern and would begin at 6:30 p.m. Transportation would begin at 6 p.m. with shuttle buses running about every 30 minutes. The last shuttle would depart from the Tavern at 9 p.m. After deducting the roughly $3,000 cost of the Tuesday evening social, Lisa expects that PDS revenues would exceed expenses by $4,000. 

Lisa mentioned that the number of participants staying at the hotel exceeded the block of rooms reserved for participants. Some people needed to switch rooms in order to accommodate everyone.  The count of total participants is 114, which includes all participants and faculty. The number participating exceeds the 96  people who participated in the Denver conference.

The PEW reimbursements have also helped to increase participation in the PDS.   Dale asked if the stipends were granted one per state or per organization in each state, since a state like Texas has three organizations. A total of 34 reimbursements will be given to active organizations, covering 26 states. Additional money from PEW will be used to offset conference expenses related to the Pew session. Kathy suggested that we send Gary VanLandingham a thank-you note for PEW providing the funding for the PDS. 

Brenda mentioned that there were quite a few new members attending the NLPES Professional Development Seminar and reminded us as executive committee members to talk with new members and tell them about the organization. Karl also asked us to watch and beware of problems or unusual things that happen during the conference. 

Naming of Primary Substantive Subcommittees
Karl made subcommittee assignments.  Karl asked: 

  • Angus to chair the Professional Development Subcommittee with Carol, Greg, and Nathalie as members,

  • Dale to chair the Communications Subcommittee with Charles and Rachel as members, and

  • Wayne to chair the Awards Subcommittee with Marcia and Lisa as members.

Subcommittee Reports

 Professional Development

Angus had three areas to cover for the Professional Development Subcommittees report: (1) the 2013 PDS, (2) the 2013 Summit, and (3) training.

2013 PDS. The PDS will be held in Austin, Tex. The plan is to team up with two other LSCC staff sections: NLSSA (Security) and the LRL (Librarians). Karl said that the Security staff section has a larger membership than the Librarians, who have a very small roster. The teaming concept has been done in the past, such as with the Baton Rouge and Denver PDSs. The concept for Austin will be to share the location, but have separate programs for each staff section. Karl said that the Sunset Commission would partner with the Legislative Budget Board and the State Auditor’s Office for the PDS.

Time frames to consider are late September and early October; however, Karl pointed out that the PDS will have to work around big events in Austin. The format for the PDS will be the same as Atlanta, which was a 2.5 day conference.

Karl asked the committee their thoughts on using Conference Direct to schedule the hotel and meeting space. Marcia commented that the government rate was lower than the rate that NLPES got through Conference Direct. Greg said that we have to look at what we get for the room rate, such as free meeting space.  It is important to remember that other costs are built into the room rate.  Lisa said there were four bids for the Atlanta PDS. This hotel was best of the four bids, because the meeting rooms were free. The only disappointment was that the breakfast at this hotel was really expensive. The audio/visual rates were not part of the original bid for the Georgian Terrace. Lisa also added that she was able to suggest hotels to Conference Direct.  Brenda added that Conference Direct has significant negotiating power. 

Action Item:  The executive committee unanimously voted to use Conference Direct in conjunction with NCSL’s staff to procure the hotel/meeting space for the Austin PDS.

Angus mentioned that the degree of involvement of the executive committee for the PDSs is driven by the preference of the host state.  Karl said that we could look at using the “grid” approach that had been used previously to identify states that had worked on particular topics and consider those for PDS panels. There was also some discussion about the host for the 2014 PDS. Brenda reminded us that the 2014 Summit will be in Minnesota. 

2013 Summit. Angus said if we want to put together a session for the 2013 Summit we need to start thinking about it now. LSCC focus is engaging issues for legislators. Brenda commented that we want to get our staff section name out there, and that we can co-sponsor sessions. She will keep tuned for possible sessions that we could help sponsor and let us know. Karl would circulate these ideas for approval of the executive committee.

Training. Angus reminded the committee that on the NLPES website in the training section, there is a matrix that lists podcasts, webinars, and other presentations by topic. Brenda stated that NCSL is changing the NCSL website—which affects the NLPES site. Data that is more than two years old is not as easily seen as in the past. Brenda said there is a box on the website to click to show information that is more than two years old.  When NCSL makes changes to the websites, it sometimes affects the NLPES website in unintended ways.

Angus said that “holes” still need to be filled in the training matrix. The PDSs provide great ideas for webinar and e-learning topics.  e could take PDS topics and adapt them to a one-hour package. Angus asked us to look for webinar ideas during the PDS. Marcia commented that it is not a good idea to have webinars on Fridays, since many members work flexible schedules and don’t work Fridays. If a member office would like to put together a webinar or a training presentation for the website, give the presentation to Angus, so his subcommittee can review and approve it for the website.

Brenda asked the committee if we would like to have our own listserv for the executive committee. She thought it might already exist, but it just needed to be activated. The subcommittees could also have their own listserv if that would be helpful. We agreed that a list serve for the full executive committee was a good idea, but the subcommittees did not need their own listservs at this time.

Brenda also asked the committee if NLPES was interested in cosponsoring an Oversight of Federal Funds webinar for broadcast this calendar year. The total cost is divided among the participating staff sections. The average cost could be $1,200 to $1,500. However, the actual cost is based on the number of connections. Kathy said that, if we participate, we need to get the actual cost in writing. Karl asked Brenda to get more information on how cost is shared among multiple webinar sponsors, and wondered if we could participate in this webinar and also do our own webinar. Angus said the subcommittee set the goal to do one webinar this year.  Brenda said she would check on the availability of funding. 


Dale reported on the Communications Subcommittee duties for the following four areas: (1) monitoring the content and format of the NLPES website, (2) publishing the NLPES newsletter, (3) improving communications with NLPES members, and (4) coordinating state contacts. 

Monitoring the NLPES Website. Dale assigned Charles to monitor the content and format of the NLPES website. Kathy said that there exists a checklist for quarterly monitoring of different parts of the website, and to check with Leslie McGuire (GA) or former executive committee members on this topic. Lisa said she would check with Leslie. At this point, the checklist likely may not be valid since the website has changed, but it could provide some guidance. Charles will have to inquire of former executive committee members to locate a copy of the checklist by the spring meeting.

Publishing the NLPES Newsletter. Dale thanked Brenda and contributors for the current edition of the newsletter. Dale created a tracking sheet to monitor who’s contributing to the four standard newsletter sections:  the Chair’s Corner, Report Radar, State Profile, and Staff Profile. There are also other sections included in the newsletter as needed, such as announcing the awards in the spring edition or upcoming events. 

The next newsletter will be released mid-November. Assignments are as follows: Chair’s Corner – Karl, Report Radar – Angus, State Profile – Wisconsin, and Staff Profile – Utah and Virginia.  There is a waiting list for the staff profile for future editions of the newsletter. Karl has a list of past state profiles and will get the list to Dale.

Improving Communication with Members. Rachel volunteered to explore ways to communicate with newer members of NLPES. Previous discussions looked at Facebook and Twitter as tools to communicate with newer members, but there also could be other communication tools that would be feasible for this organization. Nathalie said that newer members may not be aware of the NLPES listserv and could be informed to sign up. However, Kathy pointed out that some offices don’t want all their employees on the listserv and have designated specific people to receive listserve communications. Brenda mentioned that if an office changes its email protocol, to let her know and she will update it on the listserv. 

Coordinating State Contacts. Dale said that the goal is not to let some of the less active states forget about us. Brenda said she is reaching out to update key contacts. Kathy reminded the committee that several states do not have offices. Most states that have auditing/program evaluation offices stay involved with NLPES, but a few states are not as involved. It was suggested that, if members of the executive committee know someone in another state office, we should contact them, rather than make a blind call. It takes a personal contact to get people involved. 

Wayne reported on the Awards Subcommittee duties for the following two areas: (1) administration of the awards program, and (2) revision to awards criteria.

Administration of Awards Program. Wayne told the committee that Marcia would be the lead judge for the certificates of impact committee, Lisa would be the lead judge for the Excellence in Evaluation Award, and Wayne would volunteer someone from the Utah office to be the lead judge for the Research Methodology Committee.  Marcia, Lisa, and Wayne would each ask two members from other states to participate as judges, so each award would have a panel of three judges.

Wayne said that the subcommittee had discussed adding the lead judge’s contact information for each award, so potential applicants could call the lead judge if there was a question about the criteria. The subcommittee also suggested that Wayne, as the chair of the subcommittee, could contact applicants if the judges for an award needed to clarify something on an application.  Kathy didn’t believe this approach was appropriate and compared the administration of the awards to an RFP process. The criteria should be clear, so that members don’t have to call to clarify criteria, and the submitted applications should be judged as is.  It doesn’t seem to be a fair process to all applicants if one is contacted to answer a clarifying question. Scott reminded the committee the awards have a brand, and we want to be careful not to do anything to lessen that brand. Wayne said the subcommittee would reconvene and revisit these concerns.  

There was also discussion, led by Greg, about the frustration of member offices that apply but do not win an award. Those offices do not receive feedback about what to do differently on a future application. Carol made the suggestion that the winning applications should be posted. That may help give some guidance to those that don’t win an award. Dale suggested that we ask the states for permission. Greg further suggested that the permission should be built into the application process, so that, by applying, the state agrees to have its application posted if it wins. 

Revision to Awards Criteria. Wayne told the committee that the subcommittee had revised the award criteria for the research methodology award. The criteria for the award were somewhat repetitive.  Hopefully, the revised criteria will be less cumbersome and will foster more applications from other states. Karl asked Wayne to email the criteria revisions to all members of the executive committee.      

Peer Reviews
Kathy said the past chair coordinates peer reviews. She has updated the peer review guidelines that James Barber originally worked on, and passed out a copy to the committee. Kathy also handed out a spreadsheet that shows a history of the peer reviews and said she would send the spreadsheet to us. That spreadsheet shows states that are active in NLPES or have program evaluation staffs, the standards that they use, and which organization, if any, does their peer reviews (such as NSAA or NLPES). Kathy said that NLPES prefers to use managers to participate on peer reviews, because they have more experience to conduct a review.


Other Business
Karl asked the committee about where to hold the spring meeting and talked about the possibilities of Denver and Austin.  The committee members agreed that the meeting should be held in Austin to visit the site prior to the 2013 PDS. Karl also asked when would be the best time in the spring. The committee agreed the end of March would be best to work around legislative sessions.  

 Meeting was adjourned.