LIHEAP ADVISORY COMMITTEE
ON MANAGING FOR RESULTS

FY 1998 REPORT

October, 1998

Prepared by

The National Energy Assistance Directors' Association
Under Contract #ACF-979246 for the
U.S. Department of Health and Human Services
Administration for Children and Families
Office of Community Services
Division of Energy Assistance

TABLE OF CONTENTS

 

I. Introduction

II. Advisory Committee Membership

III. Key Actions

A. Development of Committee Charter
B. Review of HHS GPRA Plan
C. Inventory of Performance Goals

IV. Future Actions

Appendices

Appendix I - Minutes from Teleconferences and Face-to-Face Meetings I-

November 3, 1997
December 3, 1997
February 23, 1998
March 20, 1998
June 21, 1998
July 30, 1998
September 11, 1998
September 24-25, 1998

Appendix II - LIHEAP Performance Advisory Committee Charter

Appendix III - ACF LIHEAP GPRA Plans

FY 2000 GPRA Plan
FY 1999 GPRA Plan

 

I. Introduction

In the past, much of the focus on measuring program success in the public sector has been on the effort expended and resources dedicated to a program ("inputs") by a government agency and on the number of beneficiaries assisted ("output"), rather than on whether a program is producing the desired changes in the lives of the beneficiaries ("outcome/impacts"). Input and output measures do not necessarily indicate program success if the assistance offered is not helpful to the beneficiaries of such assistance. For this reason, there is increased emphasis on looking at the effect programs have on their beneficiaries in measuring their success. The feedback provided by such data will influence strategic planning and budgeting decisions for programs. It will also influence strategic planning and budgeting decisions for programs. This change in emphasis to measuring program success has been termed "performance measurement" or "results-oriented management."

As a first step towards establishing performance measurement for the Low Income Home Energy Assistance Program (LIHEAP), Sec. 2605(b) of the LIHEAP statute, as amended by Sec. 311(b) of the Human Services Amendments of 1994 (Public Law 103-252), required the Department of Health and Human Services (HHS) to:

• develop LIHEAP model performance goals and measures that LIHEAP grantees may use to assess their success in achieving the purposes of LIHEAP;

• consult closely with state, tribal, territorial, and local LIHEAP grantees in developing the LIHEAP model performance goals and measures; and

• make the model performance goals and measures available by November 1995 in order for LIHEAP grantees to incorporate any of the elements of the model performance goals and measures, at their option, into their LIHEAP plans for FY 1997.

After the passage of the Human Services Amendments of 1994, the Office of Community Services (OCS) within HHS' Administration for Children and Families initiated a consultative process by holding meetings and establishing working groups of state and local LIHEAP officials. The purpose of the meetings was to develop a series of proposed model goals and measures that could help states evaluate the wide variety of priorities and approaches that they use in administering LIHEAP. Based on the recommendations of the working groups, OCS issued its LIHEAP model performance goals and measures to LIHEAP grantees through LIHEAP-IM-96-2, dated 11/30/95.

The Government Performance and Results Act of 1993 (GPRA) already had established a government-wide requirement for federal agencies to develop performance goals and measures for federal programs, with the intention to use this information in making decisions on budget and appropriation levels. Federal agencies were required by GPRA to develop their first performance goals plan for implementation in FY 1999.

Five strategic goals were included as part of the FY 1999 GPRA plan developed by the U.S. Administration for Children and Families (ACF), including the goal of building healthy, safe, and supportive communities and tribes. The Low-Income Home Energy Assistance Program (LIHEAP), which is administered by the Office of Community Services' Division of Energy Assistance (DEA) within ACF, focuses upon this goal. In measuring whether LIHEAP is successful in meeting this goal, ACF proposed the model LIHEAP performance goal of targeting LIHEAP assistance to eligible households with the highest home energy needs.

In October 1997, DEA established the LIHEAP Advisory Committee on Managing for Results as a joint partnership between the states, local agencies, other program stakeholders and DEA. The Committee's task is to develop recommendations on cost-effective performance goals and measures for LIHEAP that will meet the requirements of GPRA. In addition, the Committee's task is to enhance program management practices and update the recommendations of the 1995 DEA LIHEAP working group on LIHEAP model performance goals and measures. To support the work of the Committee, DEA awarded NEADA a small purchase order in September 29, 1997.

The responsibilities of the Advisory Committee were initially defined by DEA to include the following activities:

• Develop an agenda of key implementation issues and concerns that should be addressed by the Advisory Committee.

• Identify panels needed to develop specific implementation issues including data collection and verification, and recommend potential panel members.

• Issue a report to DEA on a recommended LIHEAP performance measurement system, based on the work of the panel(s).

• Identify any additional technical assistance and collaboration needed to follow up to this Project.

The Advisory Committee is comprised of representatives from state and local LIHEAP providers, data and policy experts, and DEA staff. The work of the Advisory Committee is conducted through a series of teleconferences, face-to-face meetings, and written communications. The first Advisory Committee convened in November 3, 1997 and has since been meeting on a regular basis.

II. Advisory Committee Membership

The current membership of the Advisory Committee is:

III. Key Actions

Between November 3, 1997 and September 25, 1998 the Advisory Committee held five teleconferences and three face-to-face meetings. During this period, the Committee carried out the following activities:

A. Developed a Committee Charter.

B. Reviewed and commented on proposed LIHEAP goals and measures performance plans drafted by DEA for ACF's
    GPRA FY 2000 performance plan.

C. Conducted an inventory of states' implementation plans for LIHEAP performance goals and began developing a technical       assistance plan for the states.

Each of these major activities are summarized below.

A. Development of Committee Charter

One of the first activities of the Advisory Committee was to develop a Charter for its work. The following summarizes the main points contained in the Charter, including objectives and tasks: (A complete copy of the Charter is attached as Appendix II.)

• Identify Committee Objectives.

- Assist LIHEAP grantees in identifying and demonstrating the results they are achieving in helping low income households meet their home heating and cooling needs; and

- Use LIHEAP grantee results to tell the LIHEAP program's national story.

• Assist LIHEAP grantees in demonstrating the results they are achieving in helping low income households meet their home heating and cooling needs by:

- Identify barriers to linking data to performance measurement (e.g., data collection difficulties, data analysis).

- Identify best practices, good examples and lessons learned related to measuring performance and achieving results.

- Identify and/or develop possible solutions to barriers. Solutions could be based on possible analytical approaches (e.g., using sampling to make valid estimates of performance) or best practices/lessons learned from grantees.

- Sponsor demonstrations of proposed solutions, where appropriate and as resources allow.

- Develop and initiate ways to increase the sharing of information among LIHEAP grantees on best practices, lessons learned and successful solutions to barriers.

- Oversee development and implementation of a Technical Assistance (TA) program for LIHEAP program grantees on performance measurement which includes the following subtasks:

• Develop a draft description of a TA program that identifies the key activities necessary to assist grantees in: 1) improving their ability to plan for managing for results; 2) collecting and using data to monitor and improve performance; and 3) reporting results.

• Gather feedback from the grantee and stakeholder communities on the draft TA program and develop resource estimates.

• Work with DEA staff to identify TA funding needs and sources and to implement any such TA program, as resources allow.

• Use grantee results to help tell the LIHEAP's national story.

A copy of the Committee's proposed Charter was sent to the LIHEAP network for comment in April, 1998. In summary, comments were received from six parties and suggested the following:

• Expand references to "heating" to include "cooling";

• Explicitly state that states would not have to increase their annual performance goal once they reach a target that is consistent with their program objectives;

• Request that additional funds be made available to pay for the administrative costs associated with implementing performance goals;

• Suggest that other groups be involved in the process; and

• Address how welfare reform has affected the LIHEAP program.

B. Review of LIHEAP GPRA Plan

The Advisory Committee members met with DEA on June 21, 1998 to review DEA's draft GPRA FY 2000 plan. The plan established a goal that 85 percent of LIHEAP grantees would meet their targets for participation for households which have at least one member who is 60 years or older and/or at least one member who is 6 or under.

The FY 2000 plan also proposed to delay the implementation of the home energy burden targeting goal that was included as part of the FY 1999 GPRA plan. Under the FY 1999 plan, states would have to demonstrate how they helped to increase energy targeting as measured by the reduction of home energy burden for those with the lowest incomes and highest energy expenditures for those states using actual energy bills for their LIHEAP program. This measure is being delayed because it would be too costly for states to obtain the data and there are still several serious technical issues that need to be resolved.

The Advisory Committee reviewed the draft plan and made recommendations that resulted in the following changes in the initial draft:

• The participation goal for young children was changed to age 5 and under to conform to LIHEAP regulations.

• Language providing that measures would be used to identify the program's "success or failure" was changed to "identifying performance gaps, setting improvement goals, and measuring progress, not just measuring absolute success or failure." Language was also added to say that "we may add or amend measures in the future, especially outcome measures that can be used to help make decisions and strengthen accountability."

• Program language describing the performance measure selected was changed from "increase the availability of heating assistance to eligible households with at least one member whose health is vulnerable" to "ensure the availability of fuel assistance to eligible households with at least one member whose health is vulnerable to a home without sufficient heating or cooling." In addition, the plan was changed to emphasize cooling as well as heating concerns.

• The percentage of grantees that would be successful in meeting their goal was reduced from 85% of all grantees to 75% of those grantees that set a goal in this area. This percentage was reduced in order to provide the states with a more realistic baseline goal.

Note: Copies of the FY 1999 and FY 2000 GPRA plans are included in Appendix III.

C. Inventory of LIHEAP Model Performance Goals

Advisory Committee members determined that there was a need to conduct an inventory of the number of states that are already implementing LIHEAP model performance goals. A draft inventory completed by the Advisory Committee was distributed to the states that were attending the NEADA annual meeting on June 21, 1998 with additional requests sent to states which had not sent representatives to the meeting.

A total of 39 states responded to the request. In summary, one of the more interesting results of the inventory was an indication that at least some states are beginning to recognize the importance of performance goals in evaluating their own programs. In addition, other states appeared interested in looking at programs that were working, as well as getting additional information and technical assistance on how to implement a performance-based program.

Table 1 summarizes the responses received to each of the following questions:

• Has your state adopted goals and measures for LIHEAP?
• What caused you to develop LIHEAP goals and measures?
• Has your state adopted any of the LIHEAP model goals and measures?
• In what year will you be able to report on these measures?

As shown in Table 1, 14 of the 39 states responded that they have adopted goals and measures for LIHEAP with another eight states currently in the process of developing goals and measures. Seventeen states reported that they have not developed goals and measures. Most of the states cited multiple reasons for developing goals and measures for LIHEAP, including state requirements, agency requirements and internal management decisions.

Many of the states reported that they have adopted or are planning to adopt LIHEAP model goals and measures, as summarized below:

List of LIHEAP Model Goals and Measures Selected by State

Primary Goals

A. To target energy assistance to low income households with the highest home energy needs, taking into account both energy  burden and vulnerable household members (10 states): Arizona, California, Hawaii, New York, Ohio, Oregon, South          Dakota, Tennessee, Texas, and Wisconsin.

B. To increase energy affordability for LIHEAP recipient households (7 states): Arizona, Delaware, New York, Oregon, South Dakota, Texas, and Wisconsin.

Core Measures

The change in energy burden before and after LIHEAP assistance within program elements (heating, cooling, crisis, and weatherization) and, where possible, by fuel type (7 states): Delaware, Maine (in process of implementing), New York (anticipated use), Ohio, Tennessee, Texas, and Wisconsin.

2. Percent of households served which are a high need,@ shown on a graduated scale (6 states): Hawaii, Maine (in process of implementing), New York (anticipated use), Oregon, Tennessee, and Texas.

3. Number of households for which LIHEAP assistance avoids a loss of energy services (5 states): California, Delaware, Hawaii, New York, and Oregon.

TABLE 1: FY98 LIHEAP PERFORMANCE ADVISORY COMMITTEE INVENTORY
(Responses received from 39 states)

* under Consideration

Additional Goal

C. To increase efficiency of energy usage by low-income households (8 states): Maine, New York, North Carolina, Ohio, South Dakota, Texas, Wisconsin, and Wyoming

Core Measures

1. Number of LIHEAP recipient households weatherized, including low/no-cost energy related home repair (with estimates of energy savings to be extrapolated from units weatherized) (5 states): Maine, New York (potential use), Ohio, Texas, and Wisconsin.
2. Number of LIHEAP recipient households receiving energy counseling or education (with estimates of energy savings to be extrapolated from households receiving energy counseling/education)(3 states): Maine, Texas, and Wisconsin.

In addition to the questions summarized in the table, the following three general questions concerning the reasons contributing to their decision to implement LIHEAP model performance goals and measures were also asked as part of the survey:

• What were the reasons for adopting goals and measures?
• What were the factors that hindered the adoption of these goals and measures?
• What were the types of technical assistance that could be aid in the development performance goals?
• The answers to these questions are summarized as follows:

Reasons for Adopting Goals and Measures

States cited a number of factors influencing their decision to adopt one or more of the LIHEAP model goals and measures. The most frequent mentioned was that the state felt that the measures could have a positive impact on program evaluation and improve program effectiveness. Among the other reasons cited included were the following:

• Developed goals and measures for which the state already had existing data.
• Helped to better target households as a result of reduced funding.
• Participated in pilot with NEADA.
• Improved targeting of benefits in response to concerns that funds were being allocated on the basis of a flat benefit matrix.

Factors that Hindered the Adoption of Goals and Measures

• The following main factors were cited by states as issues that hindered their adoption of any of the LIHEAP goals and measures:
• Lack of necessary data and the difficulty and cost of obtaining data required to measure energy burden and other factors.
• Lack of administrative funds and personnel.
• Need for improved data collection systems and computer programs.

Need for Technical Assistance

• Many of the responding states requested technical assistance in order to implement performance goals and measures including:
• Information provided in workshops on how to do the performance goals, including the development of clear rules and procedures for implementation.
• Examples of states that have been successful in implementing performance goals.
• Administrative funds to cover the additional costs and staff time.
• Individualized technical support on getting started.

Development of a Technical Assistance Plan

In September, the Committee drafted a Training and Technical Assistance (T&TA) Plan. The T&TA Plan was modeled on the Office of Community Services' Results Oriented Management and Accountability (ROMA) T&TA Plan that was discussed during the September meeting.

The T&TA Plan that is currently under consideration by the members of the Advisory Committee recommends to DEA a course of action to assist local, state and national partners in developing a LIHEAP performance goals and measurement system. Key components of the TTA Plan included developing a knowledge base, sharing best practices, and providing technical assistance on an individual basis.

IV. Future Actions

The Committee will continue its work in FY 1999 under NEADA's new contract with DEA. The contract will allow NEADA to support the activities of the Committee in continuing to develop a consensus among state LIHEAP offices and other stakeholders in developing LIHEAP performance standards and in collecting baseline data needed as part of ACF's GPRA Performance Plan. In addition, the contract will allow NEADA to develop the following three reports in support of the Committee's work:

• A report will be prepared on the extent to which state LIHEAP programs have access to home energy data that will be needed in calculating the energy burden as one future LIHEAP performance measure.
• A report will be prepared on the implementation of a LIHEAP performance measurement system and the identification of the technical assistance and collaborations that will be needed to implement LIHEAP performance measures.
• A report will be prepared on recommended performance measures to evaluate the impact of a restructured competitive electric and natural gas industry on the affordability of home energy for low-income customers.

In addition, the Committee will work to develop a final T&TA Plan to assist states in implementing performance goals and measures. The Committee expects to make recommendations to DEA within the next three months on an initial plan that would respond to the technical assistance requests that were made by the states in the survey, as well as identify additional areas of assistance that could be carried out to provide a comprehensive program.

APPENDIX I

MINUTES FROM TELECONFERENCES AND
FACE-TO-FACE MEETINGS OF THE

LIHEAP ADVISORY COMMITTEE ON MANAGING FOR RESULTS

Advisory Committee on Managing for Results
Notes from November 3, 1997 Teleconference

Participants: JoAnn Choate, Thornton Ridinger, Steve Tryon, Steve Ptak, Joel Eisenberg, Roger Colton, Julie Jakopic, Patricia Steiger, Larry Kelly, Lougene March, Vickie DeKoekkoek, David Carroll, Jan Fox, Leon Litow, Hap Hadd, and Mark Wolfe.

The formal meeting of the LIHEAP Advisory committee was convened on November 3. Mark Wolfe provided introductions. Leon Litow described the background materials sent to committee members.

Hap Hadd provided a brief overview of the requirements of the Government Performance and Results Act. Hap stressed the need for the LIHEAP program to move toward a managing for results process and eventually to report measurable results on a annual basis. Mae Brooks, from the CSBG program, described the activities over the last two years of the CSBG national task force to implement a managing for results process throughout that program. The task force's efforts have included developing an agreed-to set of national goals, a menu of performance measures, a flexible reporting approach and a technical assistance initiative to help states and CAP agencies, and selling the new approach to state and local officials. Leon and Mae stated that they would pull together relevant CSBG materials and send them to Committee members.

Jan Fox and Leon noted that it was their hope that the Advisory Committee would play a role similar to the CSBG task force in moving the LIHEAP program toward a managing for results process and compliance with the GPRA requirements. Leon mentioned that, if the Committee agreed, the Model LIHEAP performance goals/measures could be the starting point for this broader effort. A number of members supported that approach - that the model goals and measures correctly capture the mission of the program, and there was no need to reinvent the wheel. Several also felt that the Committee had the obligation to start with the model.

During the remainder of the teleconference, members briefly discussed a number issues the Committee could/should address in the future:

• Agreeing on broad goals for the program and identifying a set of potential core measures.
• Reviewing the use of common definitions, recognizing that systems now in use vary in the sophistication and resources will be needed to implement compatible systems nation-wide.
• Involving other stakeholders in any effort to implement a national process
• Building flexibility into data collection and reporting
• Developing performance assessment approaches

In preparation for a second meeting, the Committee agreed to the following short-term agenda:

• To review the Model performance goals and measures for any general adjustments and against the requirements in the LIHEAP statute.
• To review case studies of 10 pilots - state experiences using the Model goals and measures.
• The Committee tentatively agreed to hold another teleconference December 3 or 4. Leon and Mark will contact Committee members to arrange a specific time and data.

Advisory Committee on Managing for Results
Notes from December 3, 1997 Teleconference

Participants: JoAnn Choate, Sue Brown, Thornton Ridinger, Steve Tryon, Steve Ptak, Joel Eisenberg, Roger Colton, Julie Jakopic, Patricia Steiger, Larry Kelly, Lougene March, Vickie DeKoekkoek, David Carroll, Jan Fox, Leon Litow, Hap Hadd, Mae Brooks, and Mark Wolfe.

The Advisory Group met via a teleconference at 2:00 p.m. EST. The group discussed a wide range of topics including the ROMA process in CSBG. A discussion followed on the role of the previous LIHEAP Performance Goals task forces and the need to build on, rather than, reinvent, the work has already been completed.

It was emphasized that the process of developing and carrying out performance goals would take several years, and possibly longer to develop data that could be used at the national level. The following short term and long term tasks were discussed for the Committee:

Short Term Tasks

• Develop a charter for the Committee: The charter would describe the Committee's role, operating philosophy and major tasks - short term and long range.

Note: the development of a charter can take a long time for review and consideration of various drafts. Rather than spend our limited time together on preparing a draft charter, I would like to propose that a group of Advisory Committee members work on a Committee charter separately, and then send the draft around to the full group for comment prior to the January meeting.

Collaborate on LIHEAP Reauthorization Process: As part of the LIHEAP reauthorization process this spring, the Committee could advise OCS on whether the current LIHEAP reporting requirements need to be changed in the LIHEAP statute to collect data under GPRA. The Committee could also advise OCS on whether the statutory limit on LIHEAP training & technical assistance funds needs to be raised to allow for sufficient T & TA in implementing a "managing for results" approach for LIHEAP.

• Collaborate on Developing Baseline Data on LIHEAP Targeting: As indicated at the first meeting of the Advisory Committee, OCS has proposed to collect baseline performance data on LIHEAP targeting as part of the Administration for Children and Families' annual GPRA Plan for FY 1999. The Committee could advise OCS on issues related to collecting and analyzing such data.

Note: NEADA will distribute shortly to the Advisory Committee its report, Development of LIHEAP Targeting Performance Measures, that can be used as a starting point for the Committee's work in this area.

• Review of similar approaches taken in related programs: Issue of standardization vs. flexibility. The Committee will review the approaches to this issue taken by other federal block grants in general as part of its fact finding functions.

• Develop principles for an HHS paper on the managing for results process: Using the Model Performance Goals and Measures as a starting point, produce a draft set of goals and measures which provide the framework for the national managing for results process. It would identify core measures with common (very similar) definitions. For measures with major definitional differences, develop acceptable common definitions and estimate resources needed to implement compatible systems nation-wide.

• Review case studies: Examine the results of on-going case studies to determine lessons learned and other factors which should be incorporated into the national process.

Long Term Goals

• Provide guidance to HHS to develop an education campaign with the states to publicize the process and draft goals and measures, and to gather input and comments from the LIHEAP network.
• Develop guidelines for the implementation of a training and technical assistance initiative in support of the managing-for-results process.

Advisory Committee on Managing for Results
Notes from February 23, 1998 Meeting in Washington, DC

Participants: Thornton Ridinger, Steve Ptak, Steve Tryon, Joel Eisenberg, Julie Jakopic, Larry Kelly, Lougene March, David Carroll, Jan Fox, Leon Litow, Hap Hadd, and Mark Wolfe.

The Committee met in Washington on February 23. The main activity of the Committee was to develop a Charter. The following summarizes the main components of the Charter:

The Low Income Home Energy Assistance Program (LIHEAP) Performance Advisory Committee is hereby established at the national level by the mutual agreement of representatives from key organizations of the LIHEAP grantee network. These organizations support a wide variety of activities and services addressing low-income energy assistance needs. Following is a description of the objectives of the Committee, key initial tasks that the Committee will undertake to achieve the objectives, and the operating philosophy established by the Committee.

Objectives

The Advisory Committee is established to achieve two specific objectives:

1. Assist LIHEAP grantees in identifying and demonstrating the results they are achieving in helping low income households meet their home heating and cooling needs; and

2. Use LIHEAP grantee results to tell the LIHEAP program's national story.

Achieving these objectives will improve the effectiveness of the LIHEAP program, provide information necessary to support appropriations requests, and meet the requirements of the Government Performance and Results Act. The Committee will use a partnership approach involving local, state and federal officials, as well as LIHEAP stakeholders that are not grant-administering agencies, to achieve these objectives. The Committee will build on grantee activities and successes to improve the reporting of results and to tell the national LIHEAP story.

Initial Tasks

To achieve its objectives, the Advisory Committee has identified an initial set of tasks, as listed below.

Objective 1 - Assist LIHEAP grantees in demonstrating the results they are achieving in helping low income households meet their home heating and cooling needs.

1. Identify barriers to linking data to performance measurement (e.g., data collection difficulties, data analysis).

2. Based on data from grantees, identify best practices, good examples and lessons learned related to measuring performance and achieving results.

3. Identify and/or develop possible solutions to barriers. Solutions could be based on possible analytical approaches (e.g., using sampling to make valid estimates of performance) or best practices/lessons learned from grantees.

4 Where appropriate and as resources allow, sponsor demonstrations of proposed solutions.

5. Develop and initiate ways to increase the sharing of information among LIHEAP grantees on best practices, lessons learned and successful solutions to barriers.

6. Oversee development and implementation of a Technical Assistance (TA) program for LIHEAP program grantees on performance measurement, which includes the following subtasks:

• Develop a draft description of a TA program that identifies the key activities necessary to assist grantees in: 1) improving their ability to plan for managing for results; 2) using data to monitor and improve performance; and 3) reporting results.
• Gather feedback from the grantee and stakeholder communities on the draft TA program and develop resource estimates.
• Work with Division of Energy Assistance staff to identify TA funding needs and sources and to implement any such TA program, as resources allow.

Objective 2 - Use grantee results to help tell the LIHEAP program's national story.

1. Inventory the States regarding implementation of the LIHEAP Model Performance Goals and Measures that were issued in November 1995. Specifically:

• Is the State using any of the model goals or measures?
• What experience and/or problems has the State had in using any of the goals or measures?
• If the State is not using certain goals or measures, why not?
• Which goals or measures could the State report on and by when?
• Does the State have any suggested modifications to the current model?
• What is the State's future plans for using the model performance goals or measures?

2. Develop and disseminate a glossary of key LIHEAP program performance terms. Explore opportunities to initiate use of standard/common definitions, while recognizing the nature of LIHEAP as a block grant program.

3. Identify and describe LIHEAP's links and relationships to other programs (e.g., assisting with energy payments may keep a family from applying for welfare).

4. Analyze how to work with LIHEAP stakeholders (e.g., energy industry representatives or low income advocacy groups) as part of the performance measurement process.

5. Collaborate on developing baseline data and eventually targets for preliminary performance measures described in ACF's FY 1999 Annual Performance Plan which has been sent to Congress, or identify alternative measures. Advise the Division of Energy Assistance on issues related to collecting and analyzing baseline performance data as described in the report, Development of LIHEAP Targeting Performance Measures, published by the National Energy Assistance Directors' Association in December 1997.

Note: A complete copy of the Charter is included in Appendix II.

Advisory Committee on Managing for Results
Notes from March 20, 1998 Teleconference

Participants: JoAnn Choate, Thornton Ridinger, Steve Tryon, Joel Eisenberg, Roger Colton, Julie Jakopic, Patricia Steiger, Larry Kelly, Lougene March, David Carroll, Jan Fox, Leon Litow, Hap Hadd, and Mark Wolfe.

The following issues were discussed:

Charter and Transmittal Note

The participants reached agreement on the Charter for the Committee following minor modifications to the draft. Several of the changes were:

• Task 5 under Objective 2 concerning the performance objectives and measures in the FY 1999 Annual Performance Plan (APP) was split into two activities for clarification; 1) Review, and if necessary, revise the initial measures now in the APP, and 2) Identify methods and timetables for collecting baseline data for the final set of measures.

• Task 3 of Objective 2 was expanded from Analyzing" to Developing and implementing a strategy for" working with LIHEAP stakeholders.

• Adding a general description of the timeframe for the Advisory Committee's currently planned tasks.

The draft transmittal note was also accepted with minor changes. The Committee decided to author the transmittal note and asked Mark Wolfe to develop an appropriate letterhead. The Division of Energy Assistance will prepare the complete package for Mark to send out to the LIHEAP grantee network.

Priority Tasks

From the Charter, the Committee identified two tasks as immediate priorities:

• developing a voluntary survey regarding implementing the LIHEAP Model Performance Goals and Measures; and
• reviewing the initial set of performance measures in the ACF FY 1999 Annual Performance Plan.

The following members volunteered for these tasks:

Note: Committee members who did not participate in the teleconference should contact Mark Wolfe to sign-up for one of the tasks.

Next Steps

The Committee agreed to have another teleconference in mid to late April. Three preliminary agenda items include:

• progress reports on the two priority tasks,

• reports from individual members on reactions from the LIHEAP grantee network to the Charter and Committee's initial set of tasks, and

• arrangements for a face-to-face Committee meeting in the May-June timeframe.

Advisory Committee on Managing for Results
Notes from June 21, 1998 Meeting
Fairmont Hotel, New Orleans

The meeting took place on June 21 at 12:00 p.m. and concluded at 5:00 p.m. The following Committee members and HHS staff participated in the meeting: David Carroll, JoAnn Choate, Roger Colton, Julie Jakopic (by speaker phone), Larry Kelly, Nieves Lopez, Lougene Marsh, Steve Ptak, Thornton Ridinger, Cathy Rowe, Steve Tryon, Jan Fox, and Mark Wolfe.

The following topics were discussed:

Proposed GPRA Plan

Jan reviewed her office's draft proposed GPRA plan for FY 2000 for LIHEAP. Jan stated that the draft would require states to set participation goals for households who have at least one member who is 60 years or older and/or at least one member who is under age 5. The plan would require that at least 85 percent of grantees would be successful in achieving their participation goals. The plan would be based on FY 1998 baseline data.

Jan also stated that her office was proposing to delay the implementation of the energy affordability goal that was included as part of the FY 1999 GPRA plan. Under the FY 1999 plan, states would have to demonstrate how their program helped to increase energy affordability, as measured by the number of grantees using actual energy bills to determine energy burden and by the targeting of LIHEAP benefits to those households with high energy burdens. The burden data requirement is being delayed because it would be too costly for states to obtain the data and there are still several serious technical issues that need to be resolved.

The Advisory Group reviewed the draft plan and made the following recommendation, which were included in the revised draft (see attached).

• The participation goal for young children was changed to age 5 and under to conform to the program regulations.

• Language providing that measures would be used to identify the success or failure of the program was changed to identifying performance gaps, setting improvement goals, and measuring progress, not just measuring absolute success or failure. Language was also added to say that we may, add or amend measures in the future, especially outcome measures that can be used to help make decisions and strengthen accountability.

• Program language describing the performance measure selected was changed from increase the availability of heating assistance to eligible households with at least one member whose health is vulnerable to ensure the availability of fuel assistance to eligible households with at least one member whose health is vulnerable to a home without sufficient heating or cooling. In addition, the plan was changed to emphasize cooling as well as heating concerns.

• Reduced the percentage of grantees that would be successful in meeting their goal from 85% of all grantees to 75% of all grantees that set a goal in this area.

Several members expressed concerns about the potential administrative costs to implement performance measures if and when they include energy burden data. Jan stated that HHS was considering requesting additional funds to cover those costs as part of its FY 1999 budget request.

Jan also stated that 1998 CPS data will not be available prior to the start of the FY 1999 program year. As a result, she stated that she has asked David Carroll to prepare an estimate from prior year CPS data that states can use to develop their FY 1999 performance goals.

Appropriations/Authorization Status

Jan and Mark reviewed the status of pending program legislation. In brief, the Senate Committee on Human Resources was scheduled to mark-up the bill during the week of July 14. The bill would contain a 5-year reauthorization, an increase in training and technical assistance funds, a GAO review of REACH, and the use of emergency funds for a broader range of purposes. Appropriations legislation was scheduled to be considered by the House during the week of July 14. The House Subcommittee on Labor, HHS, ED appropriations was expected to provide no funds for LIHEAP for FY 1999.

Inventory

Members discussed the need to conduct to an inventory of the number of states that are already implementing performance goals. A draft inventory form was completed by a subcommittee of the Advisory Committee was reviewed and then subsequently distributed to the states that were attending the NEADA meeting on June 21. The inventory included the following questions:

• Has your State developed goals and measures to assess how well the LIHEAP program achieving its program outcomes?

• What caused you to develop goals and measures?

• Has your State adopted any of the LIHEAP model goals and measures?

• In what year will you be able to report on your selected LIHEAP Model goals and Measures

• Briefly describe factors that contributed and/or hindered your State in adopting one or more of the LIHEAP Model Goals and Measures

• What technical assistance does your State need to develop a system to measure the impact of the LIHEAP program in your State?

The inventory was also distributed at a later date to all the states.

Panels

Group members discussed the need for receiving comments and suggestions from utility representatives and other interested parties. The members discussed several options including expanding the Advisory Committee membership, setting up separate panels, and holding separate meetings with interested parties. The membership decided to hold separate meetings first, and then consider expanding the Advisory committee or options at a later time.

Review of Comments

The group reviewed the comments received to date from the last Advisory Committee mailing to the HHS LIHEAP mailing list of interested parties. Mark stated that comments were received from six parties. In brief the comments suggested the following:

• expanding heating to include cooling;

• explicitly state that states would not have to increase their annual performance goal once they reach a target that is consistent with their program objectives;

• requesting that additional funds be made available to pay for the administrative costs associated with implementing performance goals;

• suggesting that other groups be involved in the process; and

• addressing how welfare reform has affected the LIHEAP program.

Date for Next Meeting

The next conference call of the Advisory Committee was scheduled for Thursday, July 30 from 2:00 p.m. to 4:00 p.m. EST. The next face to face is scheduled tentatively scheduled for September 23.

Advisory Committee on Managing for Results
Notes from July 30, 1998 Teleconference Call

The teleconference took place on July 30 at 2:00 p.m. and concluded at about 4:00 p.m. The following Committee members and HHS staff participated in the meeting: David Carroll, JoAnn Choate, Roger Colton, Jan Fox, Julie Jakopic, Larry Kelly, Leon Litow, Nieves Lopez, Lougene Marsh, Thornton Ridinger, Steve Tryon, and Mark Wolfe.

The following topics were discussed:

State Plan Requirements

Jan noted that states are encouraged, but not required, to set performance goals for vulnerable households for FY 1999. Eligible vulnerable household data will be based on an average of March CPS data for 1996, 1997, and 1998 because CPS data for 1999 will not be available until after states start their FY 1999 plan year. States will therefore have to base their eligible vulnerable population counts on data extrapolated from a three year moving average. Jan also stated that HHS has delayed the inclusion of energy burden data as part of the FY 2000 LIHEAP GPRA plan because it will be more difficult for states to obtain the data necessary to make the calculations.

Data Limitations

David discussed the limitations of the CPS data including the higher sampling error that will be found in the state level data for small population states because of the limited size of the sample. States will have to be aware of the limitation of the eligibility data when setting performance goals for FY 1999.

Several members asked about the use of RECS data for setting goals for energy burden data. David explained that the RECS data had even greater limitations than CPS data because its smaller sample size is less reliable than the CPS data for state level analysis. In addition, he stated that because it is only taken once every four years, it would not be timely for use in setting annual targets.

Home Energy Notebook

Members also discussed state-level data tables in the 1996 LIHEAP Home Energy Notebook. One member commented on the wide variation in the targeting index calculated for the states. Another was concerned that the data could be taken out of context, resulting in one state program looking relatively good while another looks relatively bad because of the percentage of the vulnerable target population that they are serving. This is because some states might target assistance to the working poor and elderly, while others might target families with young children.

It was also discussed that other factors could result in a lower rate of coverage; for example, some states might exclude residents of public housing with utilities included in the rent. David stated that it was unlikely that readers would focus on small differences; rather they would be more interested in large differences among states and the factors that would account for those differences. Even with the data limitations, none of the members objected to including the Energy Notebook in the annual report to the Congress.

There was a concern that states would be judged by how they set their performance goals. Several members commented that most states would likely be very conservative in setting their performance goals until they have a clearer understanding of how the eligibility and enrollment data will fluctuate.

National Low Income Consortium (NLIEC) Meeting

Several members commented on the performance goals workshop at the NLIEC meeting. It was stated that the meeting was well attended and demonstrated the high level of interest shown by states and local administrators in how performance goals will implemented in LIHEAP. It also showed that states are interested in having more workshops and opportunities to learn about how to implement performance goals.

Next Meeting

The next face-to-face meeting is set for September 24 from noon to 4:30 p.m. and September 25 from 9:00 a.m. to noon. A conference call was set for September 11 at 2:00 p.m. EST to plan the meeting. One of the topics for the conference call will be to discuss how to involve outside stakeholders as part of the Advisory Committee's activities.

Advisory Committee on Managing for Results
Notes from September 11, 1998 Teleconference Call

The teleconference took place on September 11 at 2:00 p.m. and concluded at 4:00 p.m. The following Committee members and HHS staff participated in the meeting: David Carroll, Jo Ann Choate, Roger Colton, Larry Kelly, Leon Litow, Lougene Marsh, Thornton Ridinger, Cathy Rowe, Steve Tryon, and Mark Wolfe. In addition, Hap Hadd joined the discussion as an observer.

Summary of Performance Goals Inventory

Mark discussed the results of the performance goals inventory. He stated that the information was first collected during the NEADA 1998 Annual Meeting held in New Orleans, with additional requests sent to states that had not submitted responses. He stated that a total of 39 states responded to the request. Fourteen of the 39 states reported that LIHEAP goals and measures have been developed with another eight states currently in the process of developing goals and measures. Seventeen states reported that they have not developed goals and measures. In addition, ten states reported that they are planning to adopt LIHEAP goals and measures in FY 1999, nine are planning to do so in FY 2000, and six are considering doing so in the future.

As reported by the inventory, states cited a number of factors influencing their decision to adopt one or more of the goals and measures. The most frequently mentioned was that the state felt that the measures could have a positive impact on program evaluation and improve program effectiveness.

The main factors cited by states as issues that hindered their adoption of any of the LIHEAP goals and measures were lack of necessary data, the difficulty and cost of obtaining data required to measure energy burden and other factors, lack of administrative funds and personnel, and need for improved data collection systems and computer programs.

Many of the responding states requested technical assistance in order to implement performance goals and measures including workshops on how to do the performance goals, examples of states that have been successful in implementing performance goals, administrative funds to cover the additional costs and staff time, and individualized technical support on getting started.

Discussion of September 24-25 Meeting Agenda

After the survey was reviewed, the Advisory Committee members began discussing the agenda for the September 24 -25 meeting in Washington. The Committee members agreed that the meeting should include the following three topic areas:

• Development of a technical assistance program to aid states in implementing performance goals

• Stakeholder's Roundtable to discuss the following topics:

a. Strategies for working with utilities and heating oil companies in order to obtain energy consumption data

b. Usefulness of performance goals data in explaining LIHEAP to the public

c. Observation on the Committee's Charter and Proposed GPRA Plan

d. Congressional interest in performance goals

• Discussion of Program Report Prepared by NEADA

• Discussion of Results Oriented Management and Accountability (ROMA) process and other HHS Initiatives on Performance Goals

Other Issues Discussed

Dissemination of Information: Several members discussed how performance goals data can and should be used to help inform the public on the positive aspects of LIHEAP and that states should be putting together ideas and models of effective performance goals systems. This information can be used to inform the LIHEAP network of what the program is accomplishing and what story the program has to tell. In addition, it was also suggested that the Committee revisit its Charter. As stated in the Charter, the Committee's tasks include: identifying barriers, developing solutions to those barriers, sponsoring demonstrations, developing ways of sharing information, overseeing the development and implementation of a technical assistance program, and using grantee results to help tell the program's national story.

GAO Glossary of Selected Terms: Larry Kelly told the committee that GAO has prepared a Glossary of Terms on Performance Goals. It was agreed that a copy of the Glossary would be provided to all Committee members.

Advisory Committee on Managing for Results
Notes from September 24-25, 1998 Meeting in Washington DC

The meeting took place on Thursday, September 24 from 1:00 p.m. to approximately 4:30 p.m. and Friday, September 25 from 9:00 a.m. to 1:00 p.m.

The following Committee members and HHS staff participated in the meeting: David Carroll, Roger Colton, Joel Eisenberg, Jan Fox, Larry Kelly, Leon Litow, Nieves Lopez, Lougene Marsh, Steve Ptak, Thornton Ridinger, Cathy Rowe, Steve Tryon, and Mark Wolfe. In addition, Hap Hadd attended as an observer.

Topics discussed on Thursday, September 24

1. Federal and State Data Requirements

Several members asked Jan Fox for an update regarding the status of HHS requirements for LIHEAP performance goals. She stated that her current understanding is that HHS cannot require states to implement performance goal and measures. HHS is, however, required by GPRA to implement national goals and measures for LIHEAP.

Jan also discussed the main differences between the FY 2000 and FY 1999 GPRA plans. Key differences included dropping the energy burden requirement and reducing the percent of states that must reach their goal for high needs households from 85 percent to 75 percent.

2. Development of Intermediate Goals

The recently released GAO report on performance goals, Balancing Flexibility and Accountability (GAO/T-GGD/HEHS-98-94) was discussed. Members were interested in any potential lessons that could be learned from the report. Of particular interest was the GAO discussion regarding the use of intermediate steps to implement performance goals. It was agreed that staff review the reports and report back to the Committee.

3. Collection of Utility Data

Strategies for collecting consumption data from utilities was discussed. Of interest was the use of utility data to quantify energy burden by household. During the discussion, several members suggested that it would be difficult, but not impossible to collect the data, provided that the utilities were interested in helping the states and have database systems that contain relevant data that can be readily retrieved.

Two major problems identified with collecting consumption data included: (1) none of the utilities are able to provide data for eligible households that are not LIHEAP recipients and (2) some utilities are only willing or able to provide data on a case-by-case basis.

4. Development of Performance Indicators

A discussion then followed regarding how to interpret performance in LIHEAP. In particular, would it be possible to develop a broader set of measures than those identified by DEA as part of its GPRA plan? Members discussed the broader measures that are frequently mentioned for LIHEAP, but are difficult to quantify. For example, one of the measures often cited is that LIHEAP helps to prevent homelessness. While some members felt that LIHEAP can be a factor in helping to prevent homelessness, others felt that the size of the grant in most states is not sufficient to make a difference of that magnitude. It was suggested that maybe one way to explain the contribution of LIHEAP was to suggest that the program makes an important incremental difference to low-income households.

5. Discussions with Stakeholders

The Committee members met with three stakeholders: Kay Guinane, National Consumer Law Center; Pat Markey, Markey and Associates; and Meg Power, National Community Action Foundation to discuss their views on a number of topics including:

• Strategies for working with utilities and heating oil companies in order to obtain energy consumption data

• Usefulness of performance goals data in explaining LIHEAP to the public

• Observation on the Committee's Charter and Proposed GPRA Plan

• Congressional interest in performance goals

Several issues were raised during the discussions:

• There is no one single approach to working with utilities and heating oil companies to obtain energy consumption data. Each company is different and uses different databases to maintain their program data. The stakeholders expressed a willingness to help approach companies to discuss the importance of collecting program data.

• Members of Congress are generally not that interested in the management aspects of LIHEAP. Members generally reference the program in terms of anecdotal cases or in reference to changing energy prices and the relative need for assistance.

• Committee members were cautioned to set goals that are realistic in light of the program's declining, or at best flat, funding level. Setting goals that are too optimistic could suggest that the program has failed.

• States should consider looking at a reduction in the number of utility shut-offs as a potential measure, in addition to the other goals that are under consideration. There has been a sharp rise in the number of disconnections during the last year as a result of utility restructuring. Performance goals in this area would not necessarily require elaborate analysis and data collection. For example, one goal could be that each state hold one meeting with their utility regulatory commission and/or utilities to address ways the LIHEAP program can more effectively interact with them in order to prevent shut-offs.

• Program recipients and "partners" such as Fuel Funds, which see clients year round, should be brought into the process now, rather than waiting until a later date.

• Because the stakeholders did not have an opportunity to review the FY 2000 GPRA plan prior to the meeting, they did not have time to comment on it in detail.

Topics discussed on Friday, September 25

1. Views on Stakeholders' Comments

The Committee members discussed the need to develop better information about LIHEAP to explain the program to the general public, in addition to improving overall program management. While they agreed in general with the stakeholders' comments that the Congress would not use performance goals data to support LIHEAP, they felt that there are other audiences that are interested in program data and can use it to explain the role of LIHEAP in helping to improve the lives of low-income households.

The members felt that inviting stakeholders to the meeting was helpful and provided a useful perspective on the work of the Committee. They agreed that stakeholders should be involved more frequently in the future so that they can provide on-going comments and suggestions as the Committee's work proceeds.

2. Update on ROMA

Julie Jacopic provided a briefing on the status of the Results Oriented Management and Accountability (ROMA) planning process. She stated that beginning in August of 1994, the Director of the Office of Community Services established a Task Force to began developing the ROMA planning process. The purpose of the process was to develop an outcome-oriented monitoring and assessment approach for the Community Services Network.

Major points raised included:

• The Task Force has established six broad goals and within each goal a number of measures reflecting the varied types of programs and services offered by CSBG to its clients. The goals included a LIHEAP placeholder as part of the goals to accommodate agencies that want to include the program as part of their ROMA plans.

• The framework for developing ROMA was to accommodate the broad range of programs and services offered by the Community Services Network. A similar system for LIHEAP would probably contain fewer components because the program is narrower in the types of services delivered .

• The Task Force has also developed a Training and Technical Assistance (TTA) plan to recommend to OCS a course of action to assist local, state and national partners in making the transition to functioning effectively within a results-oriented environment Key components of the TTA plan included developing a knowledge base, sharing best practices, and providing technical assistance on an individual basis,

2. LIHEAP T&TA Plan

The Task Force members discussed options for LIHEAP to develop a T&TA plan within the next three months similar to the ROMA plan. The plan would respond to the technical assistance requests that were made by the states, as well as identify additional areas of assistance that could be carried out to provide a comprehensive program. Mark will develop a program outline for a training and technical assistance plan prior to the October 15 teleconference, based on the ROMA T&TA plan.

3. Next Steps:

Leon stated that HHS has provided NEADA with another small purchase order to continue providing staff assistance to the Working Group. The purchase order will support the following activities:

• A report will be prepared discussing the extent to which state LIHEAP programs have access to home energy data that will be needed in calculating the energy burden as one future LIHEAP performance measure.

• A report will be prepared discussing the implementation of a LIHEAP performance measurement system and the identification of the technical assistance and collaborations that will be needed to implement LIHEAP performance measures.

• A report will be prepared on recommended performance measures to evaluate the impact of a restructured competitive electric and natural gas industry on the affordability of home energy for low-income customers.

In addition, a contract was provided to the LIHEAP Clearinghouse which had subcontracted with Response Analysis. The purpose of the subcontract is to assist four states in improving their data collection and analysis in order for them to be able to develop a performance goals and measurement system. David Carroll of Response Analysis reviewed the status of the plans for each of the states.

The meeting adjourned at about 1:00 p.m. The next teleconference for the Committee was set for 2:00 p.m., October 15.

The Low Income Home Energy Assistance Program (LIHEAP) Performance Advisory Committee is hereby established at the national level by the mutual agreement of representatives from key organizations of the LIHEAP grantee network. These organizations support a wide variety of activities and services addressing low-income energy assistance needs. Following is a description of the objectives of the Committee, key initial tasks that the Committee will undertake to achieve the objectives, and the operating philosophy established by the Committee.

The Advisory Committee is established to achieve two specific objectives:

1. Assist LIHEAP grantees in identifying and demonstrating the results they are achieving in helping low income households meet their home heating and cooling needs; and

2. Use LIHEAP grantee results to tell the LIHEAP program's national story.

Achieving these objectives will improve the effectiveness of the LIHEAP program, provide information necessary to support appropriations requests, and meet the requirements of the Government Performance and Results Act. The Committee will use a partnership approach involving local, state and federal officials, as well as LIHEAP stakeholders that are not grant-administering agencies, to achieve these objectives. The Committee will build on grantee activities and successes to improve the reporting of results and to tell the national LIHEAP story.

Initial Tasks

To achieve its objectives, the Advisory Committee has identified an initial set of tasks, as listed below.

Objective 1 - Assist LIHEAP grantees in demonstrating the results they are achieving in helping low income households meet their home heating and cooling needs.

1. Identify barriers to linking data to performance measurement (e.g., data collection difficulties, data analysis).

2. Based on data from grantees, identify best practices, good examples and lessons learned related to measuring performance and achieving results.

3. Identify and/or develop possible solutions to barriers. Solutions could be based on possible analytical approaches (e.g., using sampling to make valid estimates of performance) or best practices/lessons learned from grantees.

4. Where appropriate and as resources allow, sponsor demonstrations of proposed solutions.

5. Develop and initiate ways to increase the sharing of information among LIHEAP grantees on best practices, lessons learned and successful solutions to barriers.

6. Oversee development and implementation of a Technical Assistance (TA) program for LIHEAP program grantees on performance measurement, which includes the following subtasks:

• Develop a draft description of a TA program that identifies the key activities necessary to assist grantees in: 1) improving their ability to plan for managing for results; 2) using data to monitor and improve performance; and 3) reporting results.

• Gather feedback from the grantee and stakeholder communities on the draft TA program and develop resource estimates.

• Work with Division of Energy Assistance staff to identify TA funding needs and sources and to implement any such TA program, as resources allow.

Objective 2 - Use grantee results to help tell the LIHEAP program's national story.

1. Inventory the States regarding implementation of the LIHEAP Model Performance Goals and Measures that were issued in November 1995. Specifically:

• Is the State using any of the model goals or measures?

• What experience and/or problems has the State had in using any of the goals or measures?

• If the State is not using certain goals or measures, why not?

• Which goals or measures could the State report on and by when?

• Does the State have any suggested modifications to the current model?

• What are the State's future plans for using the model performance goals or measures?

2. Develop and disseminate a glossary of key LIHEAP program performance terms. Explore opportunities to initiate use of standard/common definitions, while recognizing the nature of LIHEAP as a block grant program.

3. Identify and describe LIHEAP's links and relationships to other programs (e.g., assisting with energy payments may keep a family from applying for welfare).

4. Analyze how to work with LIHEAP stakeholders (e.g., energy industry representatives or low income advocacy groups) as part of the performance measurement process.

5. Collaborate on developing baseline data and eventually targets for preliminary performance measures described in ACF's FY 1999 Annual Performance Plan which has been sent to Congress, or identify alternative measures. Advise the Division of Energy Assistance on issues related to collecting and analyzing baseline performance data as described in the report, Development of LIHEAP Targeting Performance Measures, published by the National Energy Assistance Directors' Association in December 1997.

Background and Rationale for this Approach

Three important concepts are affecting the way public programs do business these days. These are: 1) managing for results; 2) involving customers; and 3) empowering employees and partners.

The concept of managing for results is firmly rooted in the federal statute, the Government Performance and Results Act (GPRA). GPRA requires programs to: 1) determine and describe the outcomes they expect to achieve, and 2) manage their programs with a goal of accomplishing these results, rather than focusing exclusively on inputs and program activities. The Advisory Committee will work with LIHEAP grantees to align their activities with the planning and reporting requirements of GPRA.

The emphasis on customers is becoming integral to all government programs. Programs need to define who their customers are, and consider developing objectives and measures that address customer needs and services (e.g., responsiveness). The Advisory Committee will work with grantees to determine the program's current customer focus and, on a long range basis, may seek to stimulate customer involvement in setting the course for the LIHEAP program in the future.

With empowerment, there is a recognition