Managing For Results: Agencies' Annual Performance Plans Can Help Address
Strategic Planning Challenges (Letter Report, 01/30/98, GAO/GGD-98-44).
Pursuant to a congressional request, GAO reviewed federal agencies'
strategic plans submitted in response to the Government Performance and
Results Act of 1993, focusing on: (1) summarizing its observations on
agencies' September plans; and (2) providing additional information on
how the next phase of the Results Act's implementation--performance
planning measurement--can be used to address the critical planning issue
GAO observed in reviewing the September strategic plans.
GAO noted that: (1) on the whole, agencies' September plans appear to
provide a workable foundation for Congress to use in helping to fulfill
its appropriations, budget, authorization, and oversight
responsibilities and for agencies to use in setting a general direction
for their efforts; (2) agencies' strategic planning efforts are still
very much a work in progress; (3) GAO's reviews of September plans
indicate that continued progress is needed in how agencies address three
difficult planning challenges--setting a strategic direction,
coordinating crosscutting programs, and ensuring the capacity to gather
and use performance and cost data; (4) GAO found that agencies can build
upon their initial efforts to set a strategic direction for their
programs and activities; (5) the next stage in the Results Act's
implementation--performance planning and measurement--can assist
agencies in addressing the challenge of setting a strategic direction;
(6) as an agency develops its performance plan, it likely will identify
opportunities to revise and clarify those strategic goals in order to
provide a better grounding for the direction of the agency; (7) also, as
agencies develop the objective, measurable annual performance goals as
envisioned by the Act, those goals can serve as a bridge that links
long-term strategic goals to agencies' daily operations; (8) the Results
Act's requirements for annual performance plans and performance
measurement can also provide a structured framework for Congress, Office
of Management and Budget, and agencies to address agencies' crosscutting
programs--the second critical planning challenge; (9) GAO found that
although agencies have begun to recognize the importance of coordinating
crosscutting programs, they must undertake the substantive coordination
that is needed for the effective management of those programs; (10) the
third critical planning challenge is the need for agencies to have the
capacity to gather and use sound program performance and cost data to
successfully measure progress toward their intended results; (11) under
the Results Act, agencies are also to discuss in their annual
performance plans how they will verify and validate the performance
information that they plan to use to show whether goals are being met;
and (12) verified and validated performance information, in conjunction
with augmented program evaluation efforts, will help ensure that
agencies are able to report progress in meeting goals and identify
specific strategies for improving performance.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: GGD-98-44
TITLE: Managing For Results: Agencies' Annual Performance Plans
Can Help Address Strategic Planning Challenges
DATE: 01/30/98
SUBJECT: Strategic planning
Congressional/executive relations
Program evaluation
Federal agency accounting systems
Agency missions
Federal agency reorganization
Accountability
Internal controls
Interagency relations
IDENTIFIER: Government Performance and Results Act
GPRA
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** <info@www.gao.gov> **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Requesters
January 1998
MANAGING FOR RESULTS - AGENCIES'
ANNUAL PERFORMANCE PLANS CAN HELP
ADDRESS STRATEGIC PLANNING
CHALLENGES
GAO/GGD-98-44
Strategic Planning Challenges
(410196)
Abbreviations
=============================================================== ABBREV
BIA - Bureau of Indian Affairs
BOR - Bureau of Reclamation
CFO - Chief Financial Officer
DOD - Department of Defense
DOE - Department of Energy
DOT - Department of Transportation
EASI - Easy Access for Students and Institutions
EPA - Environmental Protection Agency
ESA - Employment Standards Administration
FAA - Federal Aviation Administration
FASAB - Federal Accounting Standards Advisory Board
FEMA - Federal Emergency Management Agency
FFELP - Federal Family Education Loan Program
FWS - Fish and Wildlife Service
GMRA - Government Management Reform Act of 1994
GPRA - Government Performance and Results Act
GSA - General Services Administration
HCFA - Health Care Financing Administration
HHS - Department of Health and Human Services
HUD - Department of Housing and Urban Development
IRS - Internal Revenue Service
ITA - International Trade Administration
MMS - Minerals Management Service
NAPA - National Academy of Public Administration
NASA - National Aeronautics and Space Administration
NOAA - National Oceanic and Atmospheric Administration
NPS - National Park Service
NRC - Nuclear Regulatory Commission
NSF - National Science Foundation
NSLDS - National Student Loan Data System
NSTC - National Science and Technology Council
OMB - Office of Management and Budget
OPM - Office of Personnel Management
OSHA - Occupational Safety and Health Administration
PTO - Patent and Trademark Office
QDR - Quadrennial Defense Review Report
SBA - Small Business Administration
SSA - Social Security Administration
SSI - Supplemental Security Income
TPCC - Trade Promotion Coordinating Committee
USAID - U.S. Agency for International Development
USDA - U.S. Department of Agriculture
VA - Department of Veterans Affairs
Letter
=============================================================== LETTER
B-278878
January 30, 1998
The Honorable Richard K. Armey
Majority Leader
House of Representatives
The Honorable John Kasich
Chairman, Committee on the Budget
House of Representatives
The Honorable Dan Burton
Chairman, Committee on Government
Reform and Oversight
House of Representatives
The Honorable Bob Livingston
Chairman, Committee on Appropriations
House of Representatives
In recent years, governments around the world, including ours, have
faced a citizenry that is demanding that government become at the
same time more effective and less costly.\1
These twin demands are the broad forces behind the move to a
performance-based approach to management in public sector
organizations--the most important effort to improve government
management in over a generation. Congress enacted the Government
Performance and Results Act of 1993, commonly referred to as "GPRA"
or "the Results Act," in conjunction with the Chief Financial
Officers (CFO) Act and information technology reform legislation,
such as the Clinger-Cohen Act of 1996, to address these twin demands
and to instill performance-based management in the federal
government.
The Results Act seeks to shift the focus of government decisionmaking
and accountability away from a preoccupation with the
activities--such as grants and inspections made--to a focus on the
results of those activities--such as real gains in employability,
safety, responsiveness, or program quality. Under the Act, agencies
are to develop strategic plans, annual performance plans, and annual
performance reports.\2 The Office of Management and Budget (OMB) is
also to prepare the annual federal government performance plan that
is based on the performance plans of individual agencies. Agencies
submitted the first cycle of the strategic plans to Congress and OMB
in September 1997. The first federal government performance plan is
to be submitted to Congress in February 1998 with the President's
fiscal year 1999 budget; and the first annual performance plans,
which were to be submitted to OMB in the fall of 1997, are due to
Congress after the submission of the President's budget.
On October 30, 1997, we submitted a statement for the record at the
request of the House Committee on Government Reform and Oversight on
those strategic plans.\3 As requested, this report builds upon that
statement to (1) summarize our observations on agencies' September
plans and (2) provide additional information on how the next phase of
the Results Act's implementation--performance planning and
measurement--can be used to address the critical planning issues we
observed in reviewing the September strategic plans.\4
--------------------
\1 See, for example, Managing for Results: Experiences Abroad
Suggest Insights for Federal Management Reform (GAO/GGD-95-120, May
2, 1995); Managing for Results: State Experiences Provide Insights
for Federal Management Reforms (GAO/GGD-95-22, Dec. 21, 1994); and
Government Reform: Goal-Setting and Performance (GAO/AIMD/GGD-95-R,
Mar. 27, 1995).
\2 Agencies are required to submit to the President and Congress
annual reports on program performance for the previous fiscal year
(the first reports for fiscal year 1999 are due by March 31, 2000)
reviewing the agencies' success in achieving the performance goals
established in their annual performance plans.
\3 Managing for Results: Building on Agencies' Strategic Plans to
Improve Federal Management (GAO/T-GGD/AIMD-98-29, Oct. 30, 1997).
\4 See Managing For Results: Critical Issues for Improving Federal
Agencies' Strategic Plans (GAO/GGD-97-180, Sept. 16, 1997) for a
discussion of the critical planning issues we first observed in the
draft strategic plans agencies used during their consultations with
Congress last summer.
SCOPE AND METHODOLOGY
------------------------------------------------------------ Letter :1
This report is based on our reviews of 24 major agencies' strategic
plans that were formally submitted to Congress and OMB by September
30, 1997. To do these reviews, we used the Results Act supplemented
by OMB's guidance on developing the plans (Circular A-11, part 2) as
criteria to determine whether the plans contained the six elements
required by the Act. As agreed, we focused our reviews on the
progress of agencies' strategic planning efforts, specifically their
efforts to improve their strategic plans, with particular attention
to the key planning challenges that are most in need of sustained
attention. Agencies included in our analysis are listed in appendix
I, and our observations on individual agencies are summarized in
appendixes II through XXV. To gather information on how annual
performance planning and measurement could be used to address the
critical planning challenges we observed in our reviews of the
September plans, we relied on our recent report on critical
challenges needing sustained attention, our report on governmentwide
implementation of the Results Act, our guidance for congressional
review of Results Act implementation, and our guidance on effectively
implementing the Act.\5
We reviewed individual agency plans from September 30, 1997, through
November 1997. Our work was conducted in accordance with generally
accepted government auditing standards. We provided a draft of this
report for comment to the Director of OMB on January 5, 1998; a
discussion of OMB's comments appears at the end of this letter. In
addition, we provided drafts of the appendices we prepared on
individual agency plans to the relevant agencies for comment. The
comments from those agencies are summarized in the relevant
appendixes.
--------------------
\5 See GAO/GGD-97-180, Sept. 16, 1997; The Government Performance
and Results Act: 1997 Governmentwide Implementation Will Be Uneven
(GAO/GGD-97-109, June 2, 1997); Agencies' Strategic Plans Under GPRA:
Key Questions to Facilitate Congressional Review (GAO/GGD-10.1.16,
May 1997); and Executive Guide: Effectively Implementing the
Government Performance and Results Act (GAO/GGD-96-118, June 1996).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :2
On the whole, agencies' September plans appear to provide a workable
foundation for Congress to use in helping to fulfill its
appropriations, budget, authorization, and oversight responsibilities
and for agencies to use in setting a general direction for their
efforts. These plans represent a significant improvement over the
draft plans we reviewed last summer. For example, we found that all
but six of the draft plans were missing at least one element required
by the Results Act, and about a third were missing two of the six
required elements. In contrast, the September plans we reviewed
contained at least some discussion of each element required by the
Act. And, in many cases, those elements that had been included in
the draft plans were substantially improved.
Nonetheless, agencies' strategic planning efforts are still very much
a work in progress. Our reviews of September plans indicate that
continued progress is needed in how agencies address three difficult
planning challenges: setting a strategic direction, coordinating
crosscutting programs, and ensuring the capacity to gather and use
performance and cost data. First, we found that agencies can build
upon their initial efforts to set a strategic direction for their
programs and activities. Specifically, many of the strategic goals
contained in the September plans did not focus on results to the
extent feasible and were not always expressed in a manner conducive
to assessing progress in terms of actual performance. Also, the
plans often did not clearly link planning elements, such as strategic
goals, objectives, and strategies. In addition, many of the
strategies were incomplete and underdeveloped in that they lacked a
discussion of how agencies would accomplish goals. For example, some
plans lacked information on resources needed to achieve goals or
failed to address critical issues, such as unreliable information
systems, that threaten agencies' ability to meet strategic goals and
objectives.
The next stage in the Results Act's implementation--performance
planning and measurement--can assist agencies in addressing the
challenge of setting a strategic direction. As an agency develops
its performance plan, which is to contain the annual performance
goals it will use to track progress toward its strategic goals, it
likely will identify opportunities to revise and clarify those
strategic goals in order to provide a better grounding for the
direction of the agency. Also, as agencies develop the objective,
measurable annual performance goals as envisioned by the Act, those
goals can serve as a bridge that links long-term strategic goals to
agencies' daily operations. For example, an annual goal that is
linked to a program and also to a long-term strategic goal can be
used both to (1) hold agencies and their program offices accountable
for achieving those goals and (2) assess the reasonableness and
appropriateness of that strategic goal for the agency as a whole. In
addition, annual performance planning can be used to better define
strategies for achieving strategic and annual performance goals.
The Results Act's requirements for annual performance plans and
performance measurement can also provide a structured framework for
Congress, OMB, and agencies to address agencies' crosscutting
programs--the second critical planning challenge. In our reviews of
the September plans, we found that although agencies have begun to
recognize the importance of coordinating crosscutting programs, it is
important that they undertake the substantive coordination that is
needed for the effective management of those programs. In a recent
report on mission fragmentation and program overlap, we noted that
Congress, OMB, and agencies can continue to use the Results Act as a
framework for ensuring that goals for crosscutting programs are
consistent and, as appropriate, that program efforts are mutually
reinforcing.\6 Likewise, Congress and OMB can use this framework,
including the OMB-prepared federal government performance plan, to
facilitate the identification of program overlap, duplication, and
fragmentation among federal agencies.
Finally, the third critical planning challenge is the need for
agencies to have the capacity to gather and use sound program
performance and cost data to successfully measure progress toward
their intended results. For example, our work has shown that the
lack of reliable, timely performance and cost data has hampered, and
may continue to hamper, agencies' efforts to identify appropriate
goals and confidently assess performance.\7 Our work has also shown
that the September plans often did not adequately cover how program
evaluations would be used to assess the contributions of programs to
goal achievement and the appropriateness of those goals. Under the
Results Act, agencies are also to discuss in their annual performance
plans how they will verify and validate the performance information
that they plan to use to show whether goals are being met. Verified
and validated performance information, in conjunction with augmented
program evaluation efforts, will help ensure that agencies are able
to report progress in meeting goals and identify specific strategies
for improving performance.
--------------------
\6 Managing for Results: Using the Results Act to Address Mission
Fragmentation and Program Overlap (GAO/AIMD-97-146, Aug. 29, 1997).
\7 GAO/GGD-97-109, June 2, 1997; and GAO/GGD-97-180, Sept. 16, 1997.
BACKGROUND
------------------------------------------------------------ Letter :3
The Results Act is the centerpiece of a statutory framework that
Congress put in place during the 1990s to help resolve the
long-standing management problems that have undermined the federal
government's effectiveness and efficiency and to provide greater
accountability for results. In addition to the Results Act, the
framework comprises the CFO Act and information technology reform
legislation, including the Paperwork Reduction Act of 1995 and the
Clinger-Cohen Act of 1996. Congress enacted the CFO Act to remedy
decades of serious neglect in federal financial management by
establishing chief financial officers across the federal government
and requiring the preparation and audit of annual financial
statements. The information technology reform legislation is based
on the best practices used by leading public and private sector
organizations to manage information technology more effectively.\8
Under the Results Act, strategic plans are the starting point and
basic underpinning for performance-based management. In our report
on agencies' draft strategic plans, we noted that complete strategic
plans were crucial if they were to serve as a basis for guiding
agencies' operations and be used to help congressional and other
policymakers make decisions about activities and programs.\9 The Act
requires that an agency's strategic plan contain six key elements.
These elements are (1) a comprehensive agency mission statement; (2)
agencywide long-term goals and objectives for all major functions and
operations; (3) approaches (or strategies) and the various resources
needed to achieve the goals and objectives; (4) a description of the
relationship between the long-term goals and objectives and the
annual performance goals; (5) an identification of key factors,
external to the agency and beyond its control, that could
significantly affect the achievement of the strategic goals; and (6)
a description of how program evaluations were used to establish or
revise strategic goals and a schedule for future program evaluations.
Building on the decisions made as part of the strategic planning
process, the Results Act requires executive agencies to develop
annual performance plans covering each program activity set forth in
the agencies' budgets.\10 The first annual performance plans,
covering fiscal year 1999, are to be submitted to Congress after the
President's budget is submitted, which is approximately February
1998. Each plan is to contain an agency's annual performance goals
and associated measures, which the agency is to use in order to gauge
its progress toward accomplishing its strategic goals. OMB is to use
the agencies' performance plans to develop an overall federal
government performance plan that is to be submitted with the
President's budget. The performance plan for the federal government
is to present to Congress a single cohesive picture of the federal
government's annual performance goals for a given fiscal year.
--------------------
\8 See Executive Guide: Improving Mission Performance Through
Strategic Information Management and Technology (GAO/AIMD-94-115, May
1994) for a discussion of the consistent best practices used by
senior managers in leading organizations.
\9 GAO/GGD-97-180, Sept. 16, 1997.
\10 The term "program activity" refers to the listings of projects
and activities in the appendix portion of the Budget of the United
States Government. Program activity structures are intended to
provide a meaningful representation of the operations financed by a
specific budget account.
SEPTEMBER PLANS INCLUDED
REQUIRED ELEMENTS AND ADDRESSED
MANY WEAKNESSES IN DRAFT PLANS
------------------------------------------------------------ Letter :4
Agencies' September plans appear to provide a workable foundation for
the continuing implementation of the Results Act. These plans
represent a significant improvement over the draft plans we reviewed
last summer.\11 In those reviews, we found that all but six of the
draft strategic plans were missing at least one required element, and
about a third were missing two of the six required elements. In
addition, just over a fourth of those plans failed to cover at least
three of the required elements. Moreover, we found that many of the
elements included in the plans contained weaknesses--some that were
more significant than others.
The agencies, on the whole, made a concerted effort during August and
September to improve their plans. For example, all of the September
plans we reviewed contained at least some discussion of each element
required by the Act. And, in many cases, those elements that
contained weaknesses were substantially improved by September. For
example:
-- The Department of Transportation explained more clearly how its
mission statement is linked to its authorizing legislation.
-- The Small Business Administration (SBA) improved its ability to
assess progress toward its strategic goals by stating when
specific performance objectives would be met.
-- The Nuclear Regulatory Commission (NRC) better explained the
scope of its crosscutting functions by identifying major
crosscutting functions and interagency programs and its
coordination with those agencies.
-- The Department of Education improved its discussion of external
factors that could affect its achievement of strategic goals by
describing agency actions to mitigate against those factors.
Appendixes II through XXV contain our observations on the progress
and remaining challenges of individual agencies' strategic planning
efforts.
--------------------
\11 See GAO/GGD-97-180, Sept. 16, 1997, for a list of our reports on
27 agencies' draft strategic plans (the 24 major agencies plus the
U.S. Trade Representative, OMB, and the U.S. Postal Service).
CRITICAL PLANNING CHALLENGES
REMAIN TO BE ADDRESSED AS
EFFORTS UNDER THE RESULTS ACT
PROCEED
------------------------------------------------------------ Letter :5
Although the September plans appear to provide a workable foundation
for the continuing implementation of the Results Act, we found that
critical planning challenges remain. Among the remaining critical
challenges are (1) clearly establishing a strategic direction for
agencies by improving goal-setting and measurement; (2) improving the
management of crosscutting program efforts by ensuring that those
programs are appropriately coordinated to avoid duplication,
fragmentation, and overlap; and (3) ensuring that agencies have the
data systems and analytic capacity in place to better assess program
performance and costs, improve management and performance, and
establish accountability. The forthcoming annual performance
planning and measurement processes offer agencies an opportunity to
make progress in addressing these challenges.
AGENCIES CAN USE THE ANNUAL
PERFORMANCE PLANNING PROCESS
TO BUILD UPON THEIR INITIAL
EFFORTS TO ESTABLISH A
STRATEGIC DIRECTION
---------------------------------------------------------- Letter :5.1
By improving on their draft strategic plans, agencies' September
plans undertook the first steps toward setting a strategic direction
for their programs and activities. However, we found that the
September plans often lacked clear articulation of the agency's
strategic direction: (1) strategic goals and objectives were not as
measurable and results oriented as possible, (2) linkages among
planning elements were not clear, and (3) strategies for achieving
those goals and objectives were incomplete or underdeveloped.\12
However, the performance planning and measurement phase of the
Results Act offers agencies an opportunity to continue to refine
their strategic directions.
In our reviews of agencies' September plans, we found that some
agencies have begun to address the challenge of setting a strategic
direction. For example:
-- The most notable improvement in the plan for the Department of
Health and Human Services (HHS) is the inclusion of an outline
of strategic objectives for accomplishing the Department's six
strategic goals. Those objectives are largely focused on
outcomes and are defined in measurable terms. This plan also
identifies for each strategic objective the key measures of
progress. For example, one measure of progress for the
outcome-oriented objective of "reducing the use of illicit
drugs" is "death rate of persons aged 15 to 65 attributed to
drug use."
-- The September plans of the Departments of Agriculture,
Education, and the Treasury now include helpful matrixes to link
various planning elements, such as goals, objectives, measures,
and programs or responsible organizational components. These
matrixes are also useful in assessing a plan's underlying logic,
determining programmatic accountability, and identifying
crosscutting programs and potential duplication and overlap
among program efforts. For example, Treasury's September plan
contained an appendix that identified which bureau or office is
responsible for achieving its Department-wide goals and
objectives.
-- The September plan for the Department of Energy (DOE) better
explains how it plans to accomplish many of its goals. The plan
provides greater specificity on the money, staff, workforce
skills, and facilities that the agency plans to employ to meet
its goals. For example, to support its national security goal,
DOE's plan says it will need to change the skills of its
workforce and how it constructs new experimental test
facilities.
Although improvements were not isolated to these agencies, we also
found that agencies need to further clarify their strategic
directions if the Results Act is to be effective in guiding the
agencies and informing congressional and other decisionmakers. The
goals and objectives of many agencies could be more results oriented
and expressed in a manner that will better allow for a subsequent
assessment of whether the goals and objectives have been achieved.
For example, the plan for the Department of Veterans Affairs (VA)
contains the following objectives supporting the goal for its
compensation and pension area: "(1) evaluate compensation and
pension programs and (2) modify these programs, as appropriate."
Also, although the first goal in the Social Security Administration's
(SSA) September plan "[t]o promote valued, strong, and responsive
social security programs and conduct effective policy development,
research, and program evaluation" sets a strategic direction for the
agency, it could be stated in more measurable terms to better enable
the agency to make a future assessment of whether it is being
achieved.
Another challenging area for agencies in setting strategic direction
in the September plans was to establish linkages among planning
elements, such as goals, objectives, and strategies. For example,
Treasury's plan says that the Internal Revenue Service (IRS) has a
role in three law enforcement objectives--to reduce counterfeiting,
money laundering, and drug smuggling. However, the IRS plan
contained no specific strategy to help achieve any of those
objectives. In another example, the September plan for the Federal
Emergency Management Agency (FEMA) included lists of objectives and
strategies under each goal with no explanation of how the strategies
would contribute to achievement of the objectives.
Another weakness of agencies' September plans was incomplete and
underdeveloped strategies for achieving long-term strategic goals and
objectives. More specifically, we found that agencies did not always
provide an adequate discussion of the resources needed to achieve
goals. For example, SBA's September plan did not contain any
discussion on the resources, such as human resources and information
technology, needed to achieve its goals. Although other plans we
reviewed discussed resources, the discussions were incomplete. For
example, few plans discussed the physical capital resources, such as
facilities and equipment, needed to achieve their goals. Although
many agencies may not rely heavily on physical capital resources,
even the plans of some of those that do, such as the General Services
Administration and the National Park Service, a component of the
Department of the Interior, did not provide a focused discussion of
their capital needs and usage.
The role that information technology played, or can play, in
achieving agencies' long-term strategic goals and objectives was
generally neglected in the September plans. The government's track
record in employing information technology is poor, and the strategic
plans we reviewed often contained only limited discussions of
technology issues. For example, most of the Department of Defense's
(DOD) strategic goals are fundamentally linked to information
technology. However, we have placed DOD's management of critical
information management processes on our high-risk list. We believe
DOD's strategic plan would be significantly enhanced if it more
explicitly linked its strategic goals to a strategy for improving
management and oversight of information technology resources.
Additionally, DOD should recognize the dramatic impact the Year 2000
problem will likely have on its computer operations, including the
mission-critical applications identified in its strategic plan.\13
The Department of State's September plan also does not specifically
address the serious deficiencies in State's information and financial
accounting systems. Rather, the plan notes, in more general terms,
that it will take State several years to develop performance measures
and related databases in order to provide sufficient information on
achievement of its long-term goals. The lack of such a discussion in
many of the plans is of particular concern because, without it,
agencies cannot be certain that they are (1) addressing the federal
government's information technology problems and (2) better ensuring
that technology acquisition and use are targeted squarely on program
results.
--------------------
\12 See GAO/GGD-97-180, Sept. 16, 1997; and GAO/T-GGD/AIMD-98-29,
Oct. 30, 1997, for background information on the critical strategic
planning issues we identified in our prior work.
\13 On January 1, 2000, many computer systems, including DOD and
defense contractor systems, if not adequately modified, will either
fail to run or malfunction simply because the equipment and software
were not designed to accommodate the change of the date to the new
millennium. See The Results Act: Observations on DOD's Draft
Strategic Plan (GAO/NSIAD-97-219R, Aug. 5, 1997).
ANNUAL PERFORMANCE
PLANNING AND MEASUREMENT
OFFERS AGENCIES
OPPORTUNITY TO BETTER
CLARIFY A STRATEGIC
DIRECTION
-------------------------------------------------------- Letter :5.1.1
Strategic planning--setting a strategic direction for agency
operations--did not end with the submission of a strategic plan to
Congress last September. Performance-based management, as envisioned
by the Results Act, is not a linear, sequential process but, rather,
an iterative one in which strategic and performance planning cycles
will result in subsequent revisions to both strategic and annual
performance plans. Each cycle of strategic planning and performance
planning, particularly in the first few years of governmentwide
implementation of the Results Act, will likely result in agencies
making significant changes and improvements in those documents.
Consequently, agencies can continue to address the critical planning
challenges associated with setting a strategic direction as they
develop their first annual performance plans. For example, the
process of defining targeted levels of performance within set time
frames and providing baselines against which to compare actual
performance will likely produce opportunities for agencies to revisit
and improve upon their strategic goals and objectives so that those
goals are as results oriented and measurable as they can be.
If successfully developed, those annual performance goals can
function as a bridge between long-term strategic planning and
day-to-day operations, thereby assisting agencies in establishing
better linkages among planning elements. For example, agencies can
use performance goals to show clear and direct relationships in two
directions--to the goals in the strategic plans and to operations and
activities within the agency. By establishing those relationships,
agencies can (1) provide straightforward roadmaps that show managers
and staff how their daily activities can contribute to attaining
agencywide strategic goals, (2) hold managers and staff accountable
for contributing to the achievement of those goals, and (3) provide
decisionmakers with information on their annual progress in meeting
the goals. As agencies gain experience in developing these annual
performance goals, they likely will become better at identifying and
correcting misalignment among strategic goals, objectives, and
strategies within their plans.
The importance of clearly showing how strategies are linked to goals
is underscored by the Results Act requirement that annual goals are
to be based on budgetary program activities. Unlike previous federal
reform initiatives, the Results Act requires agencies to plan and
measure performance using the same program activity structures that
form the basis for their budget requests. However, we have found
that the relationships among the budget structures, performance
plans, and strategic plans will require coordinated and recurring
attention by Congress, OMB, and agencies as they move to implement
the annual performance planning and measurement phase of the Act.\14
This attention is important because the wide variability of the
budget structures indicates that the suitability of those structures
for the Results Act's performance planning and measurement will also
vary.
For example, we reported in 1997 that agency officials we spoke with
confirmed the varying suitability of their program activity
structures for the Results Act's purposes.\15
One agency successfully worked through its recent
performance-planning process using its existing program activities.
A second agency had a program activity structure that reflected its
organizational units--a structure that is useful for traditional
accountability purposes, such as monitoring outputs and staff
levels--but less useful for results-oriented planning. Still other
agencies separated performance planning from program activity
structures, believing it necessary to first establish appropriate
program goals, objectives, and measures before considering the link
to the budget. These agencies planned to rely on the Results Act's
provision to aggregate, disaggregate, or consolidate program
activities in constructing their annual performance plans.
In addition, annual performance planning can be used to better define
strategies for achieving strategic and annual performance goals. For
example, annual performance plans provide agencies with another
opportunity to further discuss strategies for information technology
investments and the operational improvements expected from those
investments. The annual performance plans should also provide annual
performance measures that Congress and other decisionmakers can use
to determine if those investments are achieving the expected
improvements. Thus, annual performance planning and measurement can
provide decisionmakers with an early warning of information
investment strategies that need to be revisited.
--------------------
\14 Performance Budgeting: Past Initiatives Offer Insights for GPRA
Implementation (GAO/AIMD-97-46, Mar. 27, 1997).
\15 GAO/AIMD-97-46, Mar. 27, 1997
AGENCIES AND CONGRESS CAN
USE PERFORMANCE PLANNING TO
ADDRESS CROSSCUTTING PROGRAM
EFFORTS
---------------------------------------------------------- Letter :5.2
A focus on results, as envisioned by the Results Act, implies that
federal programs that contribute to the same or similar results
should be closely coordinated to ensure that goals are consistent
and, as appropriate, program efforts are mutually reinforcing.\16 We
have found that uncoordinated program efforts can waste scarce funds,
confuse and frustrate program customers, and limit the overall
effectiveness of the federal effort.\17
This suggests that federal agencies are to look beyond their
organizational boundaries and coordinate with other agencies to
ensure that their efforts are aligned and complementary.
Agencies' September plans show progress in this area, but
coordination of crosscutting programs continues to be a strategic
planning challenge. During the summer of 1996, in reviewing early
strategic planning efforts, OMB alerted agencies that augmented
interagency coordination was needed at that time to ensure
consistency among goals in crosscutting programs areas. However, the
draft strategic plans we reviewed during the summer of 1997 often
lacked evidence that agencies in crosscutting program areas had
worked with other agencies to ensure that goals were consistent;
strategies were coordinated; and, as appropriate, performance
measures were similar.
Agencies' September plans better described crosscutting programs and
coordination efforts. Some plans, for example, contained references
to other agencies that shared responsibilities in a crosscutting
program area or discussed the need to coordinate their programs with
other agencies. For example, as noted earlier, NRC better explained
its crosscutting functions in its September plan. In addition, the
Environmental Protection Agency's (EPA) plan contains an appendix
that lists the federal agencies with which EPA coordinated. This
appendix describes the major steps in the coordination process and
lists by strategic goal the agencies with which EPA believes greater
integration and review of efforts will be needed. Similarly, the
plan for the Department of Transportation contains a table that shows
the contributions of other federal agencies to each of its major
mission areas. NRC's, EPA's and Transportation's plans illustrate
the kind of presentation that could be especially helpful to Congress
and the administration in identifying program areas to monitor for
overlap and duplication.
These presentations, and similar ones in other agencies' September
plans that identify agencies with crosscutting programs, also provide
a foundation for the much more difficult work that lies
ahead--undertaking the substantive coordination that is needed to
ensure that those programs are effectively managed. For example, in
an improvement over its draft plan, the Department of Labor's
September plan refers to a few other agencies with responsibilities
in the area of job training programs and notes that Labor plans to
work with them. However, the plan contains no discussion of what
specific coordination mechanism Labor will use to realize
efficiencies and possible strategies to consolidate job training
programs to achieve a more effective job training system.
Our work has shown that the next phases of the Results Act's
implementation will offer a structured framework to address
crosscutting issues.\18 For example, the Act's emphasis on
results-based performance measures as part of the annual performance
planning process should lead to more explicit discussions concerning
the contributions and accomplishments of crosscutting programs and
encourage related programs to develop common performance measures.
As agencies work with OMB to develop their annual performance plans,
they can consider the extent to which agency goals are complementary
and the need for common performance measures to allow for
cross-agency evaluations. Also, the Results Act's requirement that
OMB prepare a governmentwide performance plan that is based on the
agencies' annual performance plans can be used to facilitate the
identification of program overlap, duplication, and fragmentation.
Our work also indicates that if agencies and OMB use the annual
planning process to highlight crosscutting program efforts and
provide evidence of joint planning and coordination of those efforts,
the individual agency performance plans and the governmentwide
performance plan should help provide Congress with the information
needed to identify agencies and programs addressing similar missions.
Once these programs are identified, Congress can consider the
associated policy, management, and performance implications of
crosscutting program efforts and whether individual programs make a
sufficiently distinguishable contribution to a crosscutting national
issue. This information should also help identify the performance
and cost consequences of program fragmentation and the implications
of alternative policy and service delivery options. These options,
in turn, can lead to decisions concerning department and agency
missions and the allocation of resources among those missions.\19
--------------------
\16 GAO/AIMD-97-146, Aug. 29, 1997.
\17 GAO/GGD-97-109, June 2, 1997.
\18 GAO/AIMD-97-146, Aug. 29, 1997.
\19 GAO/AIMD-97-146, Aug. 29, 1997.
PERFORMANCE PLANNING CAN
ASSIST AGENCIES IN BUILDING
THE CAPACITY TO GATHER,
PROCESS, AND ANALYZE
PERFORMANCE AND PROGRAM COST
INFORMATION
---------------------------------------------------------- Letter :5.3
Our previous work has shown that agencies need to have reliable data
during their planning efforts to set realistic goals and later, as
programs are being implemented, to gauge their progress toward
achieving those goals.\20 In addition, in combination with an
agency's performance measurement system, a strong program evaluation
capacity is needed to provide feedback on how well an agency's
activities and programs contributed to achieving its goals and to
identify ways to improve performance.\21 However, our work has also
found serious shortcomings in agencies' ability to generate reliable
and timely data to measure their progress in achieving goals and to
provide the analytic capacity to use that data. The Results Act's
requirement that annual performance plans discuss the verification
and validation of data provides agencies with an opportunity to be
forthcoming about data limitations and to show how those limitations
will be addressed. Verified and validated performance information,
in conjunction with augmented program evaluation efforts, will help
ensure that agencies are able to report progress in meeting goals and
identify specific strategies to improve performance.
The absence of both sound program performance and cost data and the
capacity to use those data to improve performance is a critical
challenge that agencies must confront if they are to effectively
implement the Results Act. Efforts under the CFO Act have shown that
most agencies are still years away from generating reliable, useful,
relevant, and timely financial information, which is urgently needed
to make our government fiscally responsible. The widespread lack of
available program performance information is equally troubling. For
example, in our June report on a survey of managers in the largest
federal agencies, we found that fewer than one-third of those
managers said that results-oriented performance measures existed to a
great or very great extent for their programs.\22
Our work also suggests that even when performance information exists,
its reliability is frequently questionable. For example, our work
has shown that the reliability of performance data currently
available to a number of agencies is suspect, because the agencies
must rely on data collected by parties outside the federal
government. In a recent report, we noted that the fact that data
were largely collected by others was the most frequent explanation
offered by agency officials for why determining the accuracy and
quality of performance data was a challenge.\23 In our June 1997
report on the implementation of the Results Act, we also reported on
the difficulties that agencies were experiencing as a result of their
reliance on outside parties for performance information.\24
Agencies are required under the Results Act to describe in their
annual performance plans how they will verify and validate the
performance information that will be collected. This section of the
performance plan can provide important contextual information for
Congress and agencies to address the weaknesses in this area. For
example, this section can provide an agency with the opportunity to
alert Congress to the problems the agency has had or anticipates
having in collecting needed results-oriented performance information.
Agencies can also use this section to alert Congress to the cost and
data quality trade-offs associated with various collection
strategies, such as relying on sources outside the agency to provide
performance data and the degree to which those data are expected to
be reliable. The discussion in this section can also provide
Congress with a mechanism for examining whether the agency currently
has the data to confidently set performance improvement targets and
will later have the ability to report on its performance.
More broadly, continuing efforts to implement the CFO Act also are
central for ensuring that agencies resolve their long-standing
problems in generating vital information for decisionmakers. In that
regard, the Federal Accounting Standards Advisory Board (FASAB) has
developed a new set of accounting concepts and standards that
underpin OMB's guidance to agencies on the form and content of their
agencywide financial statements.\25 As part of that effort, FASAB
developed managerial cost accounting standards that were to be
effective for fiscal year 1997. These standards are to provide
decisionmakers with information on the costs of all resources used
and the costs of services provided by others to support activities or
programs. Such information would allow for comparisons of costs
across various levels of program performance.
However, because of serious agency shortfalls in cost accounting
systems, the Chief Financial Officers Council--an interagency council
of the CFOs of the major agencies--requested an additional 2 years
before the standard would be effective. FASAB recommended extending
the date by 1 year, to fiscal year 1998, with a clear expectation
that there would be no further delays.
Under the Results Act, another aspect of performance planning is a
requirement for agencies to discuss the use and planned use of
program evaluations that can provide feedback on how well an agency's
activities and programs contributed to the achievement of its goals
and to assess the reasonableness and appropriateness of those goals.
However, our recent report on agencies' draft plans stated that 16 of
the 27 draft plans did not discuss program evaluations.\26 Although
all the September plans included discussions of program evaluations,
we continued to find weaknesses in those discussions. However, this
is not surprising because agencies that had not undertaken program
evaluations prior to the preparation of the first cycle of strategic
plans would not likely be able to discuss in their September plans
how they used program evaluations to help develop the plans.
Of greater concern, many agencies, including the Departments of
Health and Human Services, Justice, and Labor, also did not discuss
how they planned to use evaluations in the future to assess progress
or did not offer a schedule for future evaluation as required by the
Results Act. In contrast, the National Science Foundation's
September plan contains a noteworthy exception to this trend. The
plan discusses how the agency used evaluations to develop key
investment strategies, action plans, and its annual performance plan.
It also discusses plans for future evaluations and provides a general
schedule for their implementation.
Over the longer term, the program performance information that
agencies are to generate under the Results Act should be a valuable
new resource for Congress to use in its program authorization,
oversight, budget, and appropriation responsibilities. As we have
noted before, to be most useful in these various contexts, that
information needs to be consolidated with budget data and critical
financial and program cost data, which agencies are to produce and
have audited under the CFO Act.\27 This consolidated program
performance, cost, and budget information, in conjunction with the
annual performance plans, should provide congressional and other
decisionmakers with a more complete picture of the results,
operational effectiveness, and costs of agencies' operations.
--------------------
\20 GAO/GGD-97-109, June 2, 1997.
\21 Managing for Results: Analytic Challenges in Measuring
Performance (GAO/HEHS/GGD-97-138, May 30, 1997).
\22 GAO/GGD-97-109, June 2, 1997.
\23 GAO/HEHS/GGD-97-138, May 30, 1997.
\24 GAO/GGD-97-109, June 2, 1997.
\25 FASAB was created in October 1990 by the Secretary of the
Treasury, the Director of OMB, and the Comptroller General to
consider and recommend accounting principles for the federal
government. If accepted by Treasury, OMB, and GAO, the standards are
to be adopted and issued by OMB and GAO.
\26 GAO/GGD-97-180, Sept. 16, 1997.
\27 Financial Management: Continued Momentum Essential to Achieve
CFO Act Goals (GAO/T-AIMD-96-10, Dec. 14, 1995).
CONCLUSION
------------------------------------------------------------ Letter :6
Agencies, on the whole, made significant progress in improving their
plans during August and September 1997. The strategic plans they
formally submitted to Congress and OMB in September 1997 appear to
provide a workable foundation for the continuing implementation of
the Results Act. Nonetheless, the critical planning challenges that
we found demonstrate that the effective implementation of
performance-based management and accountability, as envisioned by the
Results Act, is still, as to be expected, very much a work in
progress.
Since performance-based management is not a linear, sequential
process but, rather, an iterative one, each subsequent strategic and
performance planning cycle can, and likely will, result in revisions
to preceding planning documents. Therefore, Congress, OMB, and
agencies' senior managers can use the next stage of performance-based
management---performance planning and measurement--to ensure that
agencies continue to address the critical planning challenges as well
as maintain momentum on the implementation of the Results Act.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :7
On January 5, 1998, we provided a draft of this report to the
Director of OMB for comment. We provided drafts of the appendixes we
prepared on individual agency plans to the relevant agencies for
comment, and the comments from those agencies are summarized in the
relevant appendixes.
On January 13, 1998, a senior OMB official provided us with OMB's
comments on this report. He generally agreed with our observations
and said that the report was a useful compilation of our work on
agencies' September strategic plans. The official also said that
this report underscores that the implementation of the Results Act
will be an ongoing, iterative process in which agencies will learn
from their initial experiences in developing strategic plans and can
then apply those lessons learned as they continue to develop
strategic planning processes. In addition, the official provided
technical comments that were incorporated in this report.
---------------------------------------------------------- Letter :7.1
As arranged with your offices, unless you publicly announce its
contents earlier, we plan no further distribution of this report
until 30 days after its issue date. At that time, we will send
copies of this report to the Minority Leader of the House; the
Ranking Minority Members of your Committees; other appropriate
congressional committees; and the Director, Office of Management and
Budget. We will also make copies available to others on request.
If you have any questions concerning this report, please contact me
on (202) 512-8676.
J. Christopher Mihm, Associate Director, Federal Management
and Workforce Issues
AGENCIES COVERED BY THIS REVIEW
=========================================================== Appendix I
Department of Agriculture
Department of Commerce
Department of Defense
Department of Education
Department of Energy
Department of Health and Human Services
Department of Housing and Urban Development
Department of the Interior
Department of Justice
Department of Labor
Department of State
Department of Transportation
Department of the Treasury
Department of Veterans Affairs
Environmental Protection Agency
Federal Emergency Management Agency
General Services Administration
Office of Personnel Management
National Aeronautics and Space Administration
National Science Foundation
Nuclear Regulatory Commission
Small Business Administration
Social Security Administration
U.S. Agency for International Development
OBSERVATIONS ON THE DEPARTMENT OF
AGRICULTURE'S STRATEGIC PLAN
========================================================== Appendix II
On July 10, 1997, we issued a report on the U.S. Department of
Agriculture's (USDA) May draft strategic plan (Results Act:
Observations on USDA's Draft Strategic Plan, GAO/RCED-97-196R).
USDA's publicly issued strategic plan was submitted to the President
and Congress on September 30, 1997. As requested, we have reviewed
USDA's September strategic plan and compared the results of our
assessment with our observations on the draft plan, as reported in
July. On October 17, 1997, we briefed your staffs on our assessment
of the September strategic plan. The key points from that briefing
are summarized herein.
SUMMARY OF KEY OBSERVATIONS FROM
OUR JULY REPORT ON USDA'S DRAFT
STRATEGIC PLAN
USDA's May 1997 draft strategic plan included a Department-wide
strategic overview as well as 30 plans for the mission areas,
agencies, and staff offices that make up the Department. We reviewed
the overview and the 16 agency plans that are directly related to
accomplishing USDA's mission and implementing its programs. We also
reviewed the plans for the offices of the Chief Financial Officer and
the Chief Information Officer.
We observed that the May draft strategic plan did not fulfill the
requirements of the Results Act. USDA's overall mission and goals
were contained in the Department-wide strategic overview; the
overview then referred the reader to the agencies' plans for
information on the six required elements. However, only one of the
agencies' plans we reviewed contained all six required elements. The
draft strategic plan also fell short in several other areas necessary
for achieving the purposes of the Results Act. Among other things,
the draft strategic plan lacked an emphasis on externally focused
goals and objectives, adequate quantifiable performance measures, and
good linkages between the agencies' goals and the Department's goals.
We also reported that we could not determine the extent to which
coordination with other federal agencies, both within and outside the
Department, occurred in the formulation of the draft strategic plan.
It was also unclear whether agencies' goals and objectives had been
assessed for duplication and complementary functions. USDA's
Department-wide strategic overview acknowledged the role of USDA
agencies that carry out similar and/or complementary functions but
did not recognize the role of other federal agencies. Many of the
agencies' plans generically recognized the roles of other federal
agencies in accomplishing their missions. However, there was little
evidence in either the Department-wide strategic overview or the
agencies' plans to suggest that the agencies coordinated with other
agencies--internally or externally--when developing their goals and
objectives.
USDA's draft plan addressed some, but not all, of the high-risk
issues and management problems we had previously identified.
Generally, information on how USDA planned to address these high-risk
issues and management problems, such as the need to reduce losses in
the farm loan program, was included as goals and objectives in the
agencies' plans. However, USDA's draft plan did not address some
management issues, such as the need to reform milk marketing orders,
improve the management of agricultural trade programs, and strengthen
financial controls under credit reform. In addition, we have
identified significant, long-standing Department-wide problems in
information technology, accounting, and financial management.
However, USDA's draft strategic plan did not adequately recognize and
address these problems. For example, the plan for the Office of the
Chief Information Officer lacked time frames and milestones and the
resources needed to accomplish the stated goals. We also noted that
it lacked an explanation of how the goals were specifically linked to
the agencies' plans.
IMPROVEMENTS MADE IN USDA'S
SEPTEMBER STRATEGIC PLAN
USDA made significant improvements in its September strategic plan.
This plan incorporates many changes that make it more responsive to
the requirements of the Results Act. The strategic plan complies
with the six elements required by the Results Act and includes many
of the key attributes necessary for a quality plan. It also includes
information on some management challenges that we identified in the
past.
While all 16 agencies' plans contain the six required elements, the
clarity of information presented varies across the plans. For
example:
-- Most of the agencies' plans have comprehensive and concise
mission statements. However, the mission statements for two
agencies' plans--concerning the Agricultural Marketing Service
and Rural Development--are stated so broadly that it is
difficult to determine what the basic purpose of the agency is
or how it differs from that of other agencies. For example, it
is unclear how the mission of the Agricultural Marketing Service
differs from the missions of the Grain Inspection, Packers and
Stockyards Administration and the Foreign Agricultural Service.
-- Most of the agencies' plans have results-oriented goals and
objectives. However, some plans--those of the Farm Service
Agency, Food and Consumer Service, Animal and Plant Health
Inspection Service, and the Forest Service--have too many goals
and objectives structured around existing programs and
activities rather than the ultimate results that these agencies
should achieve. For example, the Farm Service Agency's plan has
four goals that we believe could be combined under two that
would fulfill the agency's mission--(1) improving the economic
viability of the agriculture sector and (2) protecting the
environment.
-- All of the agencies' plans provide more detailed strategies and
improved information on the resources needed for achieving goals
and objectives, compared with the information provided in the
May draft plan.
-- All of the agencies' plans provide a detailed discussion of the
external factors beyond the control of the agency that could
affect the achievement of the goals. However, the linkages
between external factors and their impact on specific goals
could be improved in some plans, such as the plans for the four
research agencies.
-- Unlike the May draft, in which only 1 of the 16 agencies' plans
included information on the relationship between annual
performance goals and strategic goals, all of the agencies'
September plans include this information. However, the quality
of the descriptions provided in this section of the agencies'
plans varies by agency. For example, some plans, such as those
for the Food and Consumer Service, Farm Service Agency, Food
Safety and Inspection Service, and Center for Nutrition Policy
and Programs, easily allow the reader to envision how the annual
performance goals relate to the strategic goals; other agencies'
plans, such as those for the Economic Research Service and the
Risk Management Agency, are less clear.
-- Most of the agencies' plans provide greater detail than they did
in the May draft on how program evaluations were used to develop
the strategic plan and how they will be used in the future.
However, two plans--those for the Food Safety and Inspection
Service and the Agricultural Research Service--state that
program evaluations were not used to develop the strategic plan,
although information on program evaluations planned for the
future is included; and the plan for the Agricultural Marketing
Service states that program evaluations were not used to develop
the plan and are not planned for the future. While these
agencies state that they did not use formal program evaluations
when developing their plans, the information provided in the
plans indicates that the results of relevant studies and
assessments were actually used to help develop the plans--which
in our opinion meets the requirements of the Results Act.
Consequently, we believe that these agencies may be using too
narrow a definition for the term "program evaluation." According
to an August 7, 1997, letter, sent by the House Majority Leader
to the Director, OMB, program evaluations should include all
significant evaluations relevant to the development and future
assessment of an agency's plan. The letter suggested that this
definition include reviews by the Inspector General, GAO, and
others that deal with program implementation and operating
policies and practices.
Moreover, we found that many of the key attributes necessary for a
quality plan, which were missing in the May draft plan, have been
included in the September strategic plan. These include clear
linkages between the agencies' goals and their statutory authorities
as well as the Department-wide goals; a better focus on external
goals rather than internal processes (the result of a separation of
strategic goals from management initiatives); and a more complete
discussion of relevant performance measures, although some agencies
are still developing baseline information and targets. For targets
included in the plans, it is sometimes unclear whether they are
annual or 5-year targets.
Some of the management challenges facing USDA that we raised in the
past have been included in the September plan. For example, reform
of the milk marketing orders is included in the Agricultural
Marketing Service's plan as an objective. Similarly, USDA revised
its strategic plan to address certain accounting and financial
management issues that the draft plan did not adequately address.
For example, the strategic plan reflects USDA's efforts to strengthen
controls for establishing and reestimating loan subsidy costs, as
required under credit reform. Also, the strategic plan recognizes
that additional staff and resources may be needed to ensure that USDA
can accomplish the goals set out in the plan for the Office of the
Chief Financial Officer.
USDA'S STRATEGIC PLAN CAN BE
IMPROVED FURTHER
In addition to the suggestions that we have made herein to improve
the clarity of some agencies' plans, some more significant aspects of
the strategic plan could be further improved. These improvements
include (1) explaining interagency coordination for crosscutting
issues and (2) addressing previously identified management problems.
USDA's September strategic plan provides more detailed information
about other agencies--both internal and external to the
Department--that share responsibilities for achieving the stated
goals and objectives. The Department-wide strategic overview now
includes links to agencies outside of the Department that are
important partners to USDA agencies. In addition, the agencies'
plans not only identify the agencies that they coordinate and consult
with, in some cases they also identify the specific roles of these
other agencies. However, we still could not determine from the
information provided in most of the agencies' plans whether
consultations actually took place with these agencies to resolve
crosscutting issues. Moreover, we could not determine whether an
assessment of duplicative or complementary programs and activities
was performed when the agencies were developing their goals and
objectives. In addition, we found that while many agencies' plans
explain that stakeholders were consulted during the plan's
development, they usually do not clearly identify the stakeholders.
Although this information is not required to be included in the
strategic plan by the Results Act, we believe that including
information in the agencies' plans that clearly identifies all
stakeholders would be helpful.
In addition, the September plan still does not include two management
issues that we identified in the past. In particular, the Foreign
Agricultural Service's plan still does not address the numerous
problems we have identified in agricultural trade programs.
Furthermore, there is little evidence to suggest that substantial
progress has been made in addressing our concerns about information
technology. Although USDA has added time frames for completing the
14 objectives appearing in its Office of Chief Information Officer's
plan, each time frame has a completion date "through FY 2002." We are
concerned about the absence of earlier time frames, or at least
interim ones, for resolving major Department-wide information
technology problems, such as the Year 2000 issue. By establishing
such time frames, it is not clear what priority USDA is really
placing on solving its information technology problems or whether the
Department has adequate strategies for doing so. In addition,
although the Office of Chief Information Officer's plan includes a
number of goals and objectives to better manage its $1 billion in
annual investments for information technology, we remain concerned
about the lack of information in the plan on the resources needed to
accomplish these goals and objectives and how they link to the
agencies' plans.
AGENCY COMMENTS
We provided a draft of our observations on USDA's strategic plan for
the Department's review and comment. We met with USDA's Acting Chief
Financial Officer and the Director, Planning and Accountability
Division, Office of Chief Financial Officer, who told us that they
were pleased that we had recognized the significant improvements made
to the strategic plan and that the additional comments made by us
would help them as they continue to refine and enhance the plan. In
addition, USDA made the following observations:
-- USDA disagreed with our statement that program evaluations were
not used to develop the Animal and Plant Health Inspection
Service's plan. While we agree that this plan recognizes the
importance of using program evaluations to set performance
goals, it does not clearly identify how the results of program
evaluations were used to develop the strategic plan.
Consequently, we have deleted this statement from our report to
reflect the agency's comment, but we would suggest that the
Animal and Plant Health Inspection Service add language to
clarify how program evaluations were used to develop its plan.
-- USDA noted that while there is no duplication of services
between the Agricultural Marketing Service and the Grain
Inspection, Packers and Stockyards Administration and the
Foreign Agricultural Service, the mission statement of the
Agricultural Marketing Service would be clarified, in future
versions of the plan, to distinguish it from the mission
statements of the other two agencies.
-- In connection with our observation about the Food and Consumer
Service's plan having too many goals that were structured around
current programs rather than results, USDA told us that the Food
and Consumer Service had considered structuring its plan around
a smaller number of generic goals. However, the agency chose to
establish six goals corresponding to its existing programs
because it believed that a plan structured in this manner would
be more meaningful to all interested parties, including external
partners and program participants. While we agree that setting
up goals around familiar programs and activities may make the
plan easier to understand, this approach may ultimately defeat
the purpose of the Results Act--which is to require agencies to
focus on outcomes by reevaluating what they do and why they do
it. Therefore, we would suggest that the Food and Consumer
Service consider restructuring the goals in its plan around
broader outcomes rather than current programs.
-- USDA disagreed with our statement that the Foreign Agricultural
Service plan still does not address the numerous problems that
we have identified in the past relating to agricultural trade
programs. For example, USDA believes that the Foreign
Agricultural Service has addressed the concerns outlined in our
report entitled U.S. Department of Agriculture: Foreign
Agricultural Service Could Benefit From Better Strategic
Planning (GAO/GGD-95-225, Sept. 28, 1995) by including
information on agency resource allocation, overseas priorities,
and trade opportunities under the management initiatives section
of the plan. According to USDA, other issues raised by us, such
as streamlining the agency's foreign service, will be addressed
in the annual performance plan. Although we agree that some
issues that we have raised in the past can be appropriately
addressed by including them in the annual performance plan,
others cannot. Over the past decade, we have issued a series of
reports that raise serious concerns about the fundamental
operations of the Foreign Agricultural Service's export
programs, such as the Foreign Market Development Program, Market
Access Program, P.L. 480 Program, and Export Credit Guarantee
Program. We believe that solutions to these problems will
require long-term planning that has not been adequately
addressed in the strategic plan.
-- Finally, USDA stated that it did not believe that the Office of
Chief Information Officer's strategic plan had to be the medium
to address specific solutions to the individual agencies' issues
identified in previous audits. Our observations on the Office
of Chief Information Officer's plan, however, did not discuss
the need for specific solutions; rather, we noted that the plan
lacked sufficient information on time frames, resources, and how
the goals and objectives were linked to other USDA agencies'
plans. We believe that such information is essential to clearly
identify what priority USDA is placing on solving its
information technology problems and determining whether the
Department has adequate strategies for addressing these issues.
This is especially important given the Secretary of
Agriculture's May 1997 direction to subcabinet officials that
fixing USDA's long-standing, pervasive information technology
management problems must be a top priority. USDA also disagreed
with our statement that there is a perceived lack of attention
on the Year 2000 issue. While we recognize that the plan
discusses the Year 2000 issue, we are concerned about the stated
time frames for completion for this objective. By stating a
"through FY 2002" completion time frame for the Year 2000
problem, we believe that the plan does not present an adequate
strategy for resolving one of USDA's most pressing information
technology management problems and one that must be solved
within the next 2 years.
ISSUE AREA CONTACT
Robert A. Robinson, Director, Food and Agriculture Issues;
Resources, Community, and Economic Development Division, (202)
512-5138.
OBSERVATIONS ON THE DEPARTMENT OF
COMMERCE'S STRATEGIC PLAN
========================================================= Appendix III
On July 14, 1997, we issued a report on the Department of Commerce's
draft strategic plan (The Results Act: Observations on Commerce's
June 1997 Draft Strategic Plan, GAO/GGD-97-152R). Commerce's
formally issued strategic plan was submitted to OMB and Congress on
September 30, 1997. As requested, we reviewed Commerce's revised
strategic plan, compared it with the earlier draft version that we
reported on in July, and identified significant changes or
improvements that Commerce made in the areas covered by our July
report and areas or required plan elements where additional
improvements still could be made as the plan evolves. We briefed
your staffs on our findings on October 17, 1997. Our findings are
summarized herein.
SUMMARY OF KEY OBSERVATIONS FROM
OUR JULY REPORT
Commerce's draft strategic plan was inadequate and incomplete in
several respects. Of the six elements required by the Results Act,
four were included in the draft plan--a mission statement, goals and
objectives, strategies for achieving goals and objectives, and a
discussion of key external factors--but each of these had weaknesses,
some more significant than others. For example, the mission
statement included the core functions of the Department and mentioned
the role of businesses and universities but not the important role
also played by other government entities. While there were useful
linkages among themes, goals, objectives, and responsible components,
the goals and objectives were not as results oriented as they could
be. The draft plan identified the Commerce bureau responsible for
each goal and objective but did not adequately discuss strategies for
achieving those goals and objectives or include required information
describing the operational processes, staff skills, and technologies,
as well as the human, capital, information, and other resources
needed to achieve them. Many but not all key external factors were
discussed, but the factors that were identified appeared to be used
to justify programs rather than to show how those factors could
affect the achievement of goals.
Commerce's draft strategic plan did not explicitly discuss the other
two elements required by the Results Act--the relationship between
long-term goals and objectives and annual performance goals and the
description of program evaluations used to establish general goals
and objectives and a schedule for future program evaluations. The
draft plan said that relating long-term goals and objectives to
annual performance goals will more appropriately be done in the
Department's future annual budget requests. The draft plan made
limited references in various sections to a few past studies of
Commerce programs, but those references did not describe how the
studies were used to establish general goals and objectives, and the
draft plan did not provide a schedule for future program evaluations.
Concerning other plan components, the draft plan provided much useful
information on Commerce's statutory authorities. However, the draft
plan could have been more useful to Commerce, Congress, and other
stakeholders if it had provided a more explicit discussion of
crosscutting activities, the major management challenges the
Department faces, and the Department's ability to provide reliable
financial and other management and program information to measure
achievement of its goals.
IMPROVEMENTS MADE IN COMMERCE'S
STRATEGIC PLAN
Commerce's publicly issued strategic plan incorporated improvements
in several areas and now addresses, to some extent, all of the
elements required by the Results Act. The improvements that the
Department made are steps in the right direction and address some but
not all of the weaknesses discussed in our July 1997 report on an
earlier draft of the plan. The plan's discussions of strategic goals
have been expanded to briefly indicate Commerce's strategy for
achieving each goal. For example, under the theme of keeping America
competitive with cutting-edge science and technology, the National
Oceanic and Atmospheric Administration (NOAA) has a goal to "predict
and assess decadal to centennial change." The plan now describes how
NOAA will approach this goal by addressing questions dealing with air
quality, ozone depletion, greenhouse warming, and climate change.
Also, the plan now more explicitly acknowledges the need to link
strategic goals and objectives to annual performance goals and
includes an illustrative performance measure for each of the
objectives under the three strategic themes. For example, the
illustrative performance measure for two of the Patent and Trademark
Office's (PTO) objectives is "reduced pendency time." This
illustrative performance measure is one of several that addresses
PTO's goal of granting exclusive rights for limited times to
inventors for their discoveries. Similarly, the plan's three
strategic theme chapters now more strongly emphasize the importance
of external factors that could affect achievement of Commerce's
strategic goals and identify more key external factors. Under the
economic infrastructure strategic theme, for example, the plan now
includes a reference to the International Trade Administration's
(ITA) strategy to identify obstacles to U.S. exports and plans for
removing such obstacles and marshaling U.S. government resources to
eliminate barriers.
Commerce's revised strategic plan includes new sections on program
evaluations, interagency linkages, and major management challenges.
The new section on the role of program evaluations discusses current
evaluations as well as future evaluation plans, provides examples,
highlights the difficulties in specifying the level and focus of
future evaluations because of year-to-year competition for funds, and
states that future evaluations for many Commerce bureaus will be
included in annual performance plans and budgets. The new section on
interagency linkages acknowledges the importance of close interagency
ties and emphasizes the Department's commitment to strengthen those
ties by reaching out to other federal agencies with complementary
responsibilities. In addition, the partnership sections of the three
strategic theme chapters now more fully identify and discuss
Commerce's shared mission responsibilities with other federal
agencies. Under the economic infrastructure theme, for example, the
plan now emphasizes those aspects of Commerce's mission that are
complementary. It points out that Commerce chairs the Trade
Promotion Coordinating Committee (TPCC), a 20-member interagency task
force charged by the President and Congress with developing and
implementing the National Export Strategy.
The new section on management challenges recognizes and discusses
three of the key management challenges facing the Department that
were highlighted in our July report--weather service modernization,
Census 2000, and financial management systems. Finally, the
usefulness of the plan has been improved by the addition of an index
or matrix, which shows which Commerce bureaus are responsible for
which strategic themes and goals; and an appendix, which provides
clearer and more comprehensive information on, and consolidates in
one place in the plan, the statutory and other authorities for the
Department and its bureaus, themes, and goals and objectives.
COMMERCE'S STRATEGIC PLAN CAN BE
FURTHER IMPROVED
While the overall quality of Commerce's strategic plan has been
improved since we reported in July 1997 on an earlier draft of the
plan, further improvements still could be made in each of the
elements required by the Results Act. As we indicated in our July
report, the mission statement could be made more complete by
explicitly recognizing that several other federal agencies as well as
state and local governments also play major roles in the areas
covered by Commerce's three strategic themes. In the export controls
area under the economic infrastructure theme, for example, the plan
acknowledges that Commerce shares mission responsibilities with the
Departments of Defense, Energy, and State and the Arms Control and
Disarmament Agency, but the mission statement does not recognize this
or other shared responsibilities.
Similarly, the treatment of crosscutting functions could clarify
Commerce's role in the three strategic theme areas, specify how the
Department's efforts intersect with or complement the efforts of the
other participants, and identify which other government entities
Commerce coordinated its plan with and the results of that
coordination. The Department's September 30, 1997, letter
transmitting the revised plan to Congress said that Commerce
consulted with stakeholders, provided them and congressional
committees with copies of its draft plans, and responded to
stakeholder and congressional comments. According to Commerce's
transmittal letter, there were no unresolved contrary views
concerning its plan.
The strategies for achieving each strategic goal could be further
expanded to specify how Commerce will hold its bureaus and managers
accountable for meeting strategic goals and the resources that will
be required to meet them. The linkages between long-term strategic
goals and objectives and annual performance goals could be improved
by (1) making the illustrative performance measures more outcome
oriented, such as by using the "number of counseling sessions" as a
measure of ITA's economic infrastructure objective to "increase trade
assistance targeted to small and medium-sized businesses," or (2)
showing how the performance measures that were added cause results.
The discussion of external factors could identify and discuss more
key factors beyond Commerce's direct control that could affect
achievement of its goals, such as congressional concerns about the
Census Bureau's plans for conducting Census 2000, and specify how the
external factors that are identified will be addressed or mitigated.
The discussion of program evaluations could indicate more
specifically how evaluations were used to establish goals/objectives
and performance measures.
Finally, the discussions of Commerce's major management challenges
and its capacity to provide reliable data on performance could
acknowledge and discuss more of the major management challenges and
data capacity problems that we emphasized in our July 1997 report,
such as managing modern information technology and the "year 2000
computer problem." Also, the plan could relate identified management
challenges, including performance measurement limitations, to
Commerce's strategic goals and objectives, discuss their implications
for achievement of its strategic goals and objectives, and indicate
more specifically how and when the Department expects to overcome
these challenges. The plan could be made more useful to stakeholders
and would better meet the intent of the Act if it identified and
discussed these types of problems as well as other material
weaknesses or high-risk areas, such as NOAA's fleet for acquiring
marine data, that are disclosed in Commerce's Federal Managers'
Financial Integrity Act reports or financial statements.
OTHER OBSERVATIONS ON COMMERCE'S
STRATEGIC PLAN
Given the diversity of its programs and activities and its bureaus'
independence, Commerce faced an especially formidable challenge in
developing its strategic plan. The Department developed a "thematic"
strategic plan that covers its major functions and activities; is
consistent with relevant statutory and other authorities; and
addresses, to some extent, the various elements required by the
Results Act. The plan's readability, usefulness, and overall
effectiveness as a planning and oversight tool could be enhanced by
streamlining its organization and content to eliminate many of the
details that do not relate directly to the Act's requirements, thus
reducing its 178-page length.
AGENCY COMMENTS AND OUR EVALUATION
We provided a copy of a draft of this briefing document to the
Department of Commerce for review and comment. On October 17, 1997,
the Director for Budget, Management and Information and Deputy Chief
Information Officer provided us with written comments. He
characterized our review as balanced and fair and said that the
Department clearly agrees that it needs to do more planning with
other agencies and crosscutting programs and that this is a very high
departmental priority. In this regard, he said that the Department
has stepped forward as the lead agency to link with the National
Academy of Public Administration (NAPA) in forming the Performance
Consortium and that a dozen other federal departments and agencies
have joined Commerce in this effort to develop common planning
activities and elements.
The Director also said that the Department disagrees with our
suggestion that its plan could be improved by providing additional
information in certain areas and eliminating many of the details that
do not relate directly to the Act's requirements. He said that the
Department made a specific decision to have a single, integrated
strategic plan that covers all its bureaus. The Department believes
that its plan demonstrates clearly how the Commerce bureaus fit
together and provide critical service to the nation and that it
addresses some of the administration's key priorities and secures the
buy-in of its bureaus. As Commerce's strategic plan evolves, we
continue to believe that its readability and specificity could be
improved by streamlining its organization, content, and presentation.
ISSUE AREA CONTACT
L. Nye Stevens, Director, Federal Management and Workforce Issues;
General Government Division, (202) 512-8676.
OBSERVATIONS ON THE DEPARTMENT OF
DEFENSE'S STRATEGIC PLAN
========================================================== Appendix IV
On August 5, 1997, we issued a report on DOD's draft strategic plan
(The Results Act: Observations on DOD's Draft Strategic Plan,
GAO/NSIAD-97-219R). The Department of Defense's formally issued
Results Act strategic plan was submitted to OMB and Congress on
September 30, 1997. As requested, we reviewed the strategic plan and
compared its changes to DOD's draft plan.\1 On October 17, 1997, we
briefed your staffs on our further observations on the strategic
plan. The key points from that briefing are summarized herein.
SUMMARY OF KEY OBSERVATIONS FROM
OUR AUGUST REPORT
Our prior evaluation revealed that DOD's draft plan included
discussions of each of the six critical components required in
strategic plans but that some were of higher quality than others. We
noted, for example, that DOD's draft plan contained a succinct
mission statement and general goals and objectives that cover its
major functions and operations and reflect its broad statutory
defense responsibilities, but it did not include schedules for
initiating and completing significant actions to achieve its goals.
We also noted that, although DOD included some discussion of other
elements, such as formidable management problems, these discussions
could be more complete. And, we suggested several improvements to
the draft plan, including that DOD (1) more completely state
strategies for achieving its goals and include schedules of
significant actions; (2) link and discuss how external factors could
affect its ability to achieve its goals; (3) discuss how program
evaluations were used in developing its goals and identify key issues
for future evaluations; (4) discuss planned or ongoing actions to
resolve persistent management problems, including time frames and
required resources; and (5) identify and discuss coordination efforts
for programs that crosscut with other agencies' programs. Finally,
we suggested that DOD develop one clear and succinct document to
serve as its strategic plan.
IMPROVEMENTS MADE IN THE STRATEGIC
PLAN
DOD revised its general goals and objectives to provide a clearer
presentation. In line with the revision, DOD also rearranged its
description of how the performance goals it is developing will be
related to the general goals in an effort to improve the
description's clarity. It also defined some terms and included some
additional information in the rearranged description. DOD's general
goals and objectives as reworded still cover its major functions and
operations and reflect its broad statutory defense responsibilities.
DOD also included a table listing the major management problems that
we have identified as high-risk areas and the documents, such as the
DOD Logistics Strategic Plan, that address each of the high-risk
areas. However, it did not include an explanation of what actions
will be taken to address the high-risk areas and when the problems in
these areas are expected to be corrected. Additionally, DOD did not
adopt our other suggested improvements, nor did it consolidate the
strategic plan into one succinct document.
STRATEGIC PLAN CAN BE FURTHER
IMPROVED
We believe that DOD's strategic plan could be further improved by
adopting the suggestions we made in our August 5, 1997, report
(summarized herein). We believe that addressing these areas would
provide decisionmakers and stakeholders the information necessary to
ensure that DOD has well-thought-out strategies for resolving ongoing
problems, achieving its goals and objectives, coordinating
crosscutting activities, and becoming more results oriented, as
expected by the Results Act.
AGENCY COMMENTS
DOD officials reiterated that the QDR, alone, has been the
Department's finalized strategic plan since it was issued in May
1997. They also stated that they included a table listing the
underlying plans that address the high-risk management problems noted
in our August 5 report but did not attach or include significant
detail from the underlying plans because that would have made their
submission too voluminous. They said that they did not include
details in summary fashion because that would not have provided
enough information. They noted that DOD is working to address its
management problems and said that those interested in seeing how the
problems are being addressed should read the underlying plans.
Additionally, the officials noted that although coordination of
programs and activities that crosscut other agencies' programs are
not discussed in DOD's strategic plan, DOD coordinates and cooperates
extensively with other federal agencies as part of its ongoing
strategic planning process. Finally, DOD officials commented that in
congressional consultations, the only change suggested was that DOD
reword a couple of its general goals and objectives.
ISSUE AREA CONTACT
David R. Warren, Director, Defense Management Issues; National
Security and International Affairs Division, (202) 512-8412.
--------------------
\1 For our August 5 report, we reviewed DOD's Quadrennial Defense
Review Report (QDR), issued in May 1997, and DOD's June 23, 1997,
letter, which explained how it believes the QDR meets Results Act
strategic planning requirements. DOD was required to perform the QDR
by section 923 of the National Defense Authorization Act for Fiscal
Year 1997 and intended it to be a fundamental and comprehensive
examination of America's defense needs from 1997 to 2015, including
potential threats, strategy, force structure, readiness posture,
military modernization programs, defense infrastructure, and other
elements of the defense program. For this report, we reviewed the
QDR and DOD's September 30, 1997, letter, which updates how DOD
believes the QDR meets Results Act strategic planning requirements.
DOD, however, considers the QDR, alone, to be its strategic plan.
Including the additional documents in our reviews gave DOD more
credit toward meeting requirements and expectations for strategic
plans than would have resulted from considering the QDR alone.
OBSERVATIONS ON THE DEPARTMENT OF
EDUCATION'S STRATEGIC PLAN
=========================================================== Appendix V
On July 18, 1997, we issued a report on the Department of Education's
draft strategic plan (The Results Act: Observations on the
Department of Education's June 1997 Draft Strategic Plan,
GAO/HEHS-97-176R). Education's formally issued strategic plan was
submitted to OMB and Congress on October 3, 1997. As requested, we
have reviewed the publicly issued strategic plan and compared it with
the observations in our July 18 report. On October 16, 1997, we
briefed your staffs on further observations on the strategic plan.
The key points from that briefing are summarized herein.
SUMMARY OF KEY OBSERVATIONS FROM
OUR JULY REPORT
The Department's June 17, 1997, draft plan generally complied with
the Results Act. Overall, it is a useful document and included all
but one of the six elements required by the Act--it did not discuss
how the agency's long-term goals and objectives will be related to
its annual performance goals. The plan's long-term goals and
objectives were succinct and logically linked to its mission
statement, and the quality of the goals and objectives reflected the
Department's thoughtful deliberation in its efforts to comply with
the Results Act. In addition, the plan addressed in some form all of
the Department's major statutory responsibilities.
Although the plan presented a logical and fairly complete description
of how the Department intends to achieve its mission, we identified a
few areas in the draft plan that could be improved. We observed that
the plan could benefit from more information, clarity, and context in
some of its components. The plan should have included an explanation
of the relationship between its long-term goals and objectives and
its annual performance goals as well as a complete description and
schedule of program evaluations. It could also have better addressed
the Department's major statutory responsibilities.
The Department has the primary responsibility for implementing
federal education policy and programs, but several other federal
agencies also provide education-related programs and services. In
our past work, we have identified opportunities for consolidating
programs in certain areas, such as job training and early childhood
education, to eliminate inappropriate duplication. The draft
strategic plan did a good job of identifying crosscutting program
activities in elementary and secondary programs, but it did not
identify or discuss activities for postsecondary programs that
require coordination. By discussing the agencies and activities
involved with the Department's higher education programs, the
strategic plan could provide Congress with a more complete picture of
the scope of the Department's coordination activities.
In its discussion of core strategies for achieving its strategic
goals and objectives, the Department identified several management
challenges it will face in the coming years, but it provides little
detail about these challenges and how it will meet them. This type
of information could help the Department and its stakeholders
identify major management problems that could impede the Department's
efforts to achieve its goals and objectives. Further, stakeholders
could benefit from knowing what the Department has done, is doing, or
plans to do to address such problems.
IMPROVEMENTS MADE IN THE STRATEGIC
PLAN
The Department's strategic plan issued October 3, 1997, included
several significant improvements that make it more responsive to the
requirements of the Results Act than its draft plan. The
Department's plan now addresses all six elements required by the
Results Act. The plan addressed the relationship between the
agency's long-term goals and objectives and its annual performance
plan--the only element missing from its draft plan--by including a
matrix linking long-term goals and objectives in the strategic plan
with fiscal year 1997 appropriation information and agency programs.
The matrix indicated where programs have a significant number of
activities or products supporting an objective. Though the strategic
plan does not specifically describe how the Department intends to
measure the performance of its programs each year, the matrix and the
supplemental information on the Department's performance indicators
(shown in appendix A of the plan) provided a better understanding of
the relationship between the Department's strategic and annual
performance plans. The Department states that the strategic plan was
based, in part, on objectives and indicators in draft program
performance plans prepared for key programs in the winter of 1997.
According to the plan, the annual performance plan (which includes
budget and performance plans for each of the Department's programs)
will further clarify this linkage.
The Department's October 1997 plan also provided a description of the
program evaluations and assessments that were used to develop each of
its four strategic goals as well as evaluations that will help to
"inform the implementation" of the plan and provide data for the
performance indicators supporting the goals. For example, at the end
of the narrative describing Goal 2 (build a solid foundation for
learning for all children), the plan stated that early evaluations of
the Even Start program and crosscutting evaluations of Goals 2000 and
the reauthorized elementary and secondary education programs were
used to develop this goal. In addition to the evaluations
highlighted in the introduction of each goal, appendix B of the plan
described 57 key program evaluations and other studies, including
information on when the evaluation data were or will be collected
and, in many instances, how often the data will be collected in the
future. The Department's strategic plan also identified agency
efforts that will help to avoid duplication among its evaluation
efforts and reduce respondent burden.
In addition, the narrative supporting the Department's mission
statement now encompasses the Department's major statutory
responsibilities. Our review of the Department's draft plan
indicated that it had failed to address the agency's statutory
requirements for basic education for adults, vocational
rehabilitation, education of individuals with disabilities, and
school-to-work opportunities. In its October 1997 plan, the
Department addressed this weakness by including as one of its key
agency functions "providing grants for literacy, employment, and
self-sufficiency."
The plan more clearly addressed the Department's civil rights
function within the goals and objectives sections. During our review
of the Department's draft plan, we observed that the agency's civil
rights function, although reflected in its mission statement, was not
addressed in the plan's long-term goals or objectives. In support of
two objectives related to goals 1 and 4, the Department added
strategies for addressing its civil rights function. Objective 1.5
is to get families and communities fully involved with schools and
school improvement efforts. In support of this objective, the plan
states that the Department will create collaborative partnerships
among parents, community groups, and other stakeholders that ensure
equal educational opportunity, and provide civil rights training and
technical assistance to build these linkages. Objective 4.2 is to
provide Education's partners the support and flexibility they need
without diminishing accountability. In support of this objective,
the plan adds the following strategy: to build civil rights
partnerships to achieve shared civil rights objectives and secure
timely improvements for students.
As required by the Results Act, the Department described in its draft
plan several factors outside the agency's program scope and
responsibilities that could negatively affect its ability to achieve
its strategic goals. The Department strengthened this discussion in
the plan by describing agency actions intended to mitigate against
seven key external factors that could affect the achievement of its
long-term goals. For example, the plan states that school systems
will need to undertake long-term investments in professional
development and other capacity-building activities if education
reforms are to succeed. Yet, pressures outside of the Department's
control may encourage school systems to focus instead on
demonstrating short-term gains. To counter these pressures, the plan
states that the Department will (1) work with program and technical
assistance providers to highlight the importance of sustained
professional development aligned with the standards and (2) emphasize
the importance of professional development in its performance
indicators.
Consistent with our suggestions in our July report, the Department's
strategic plan also addressed several other issues. The plan
specifically identified coordination activities related to the
Department's postsecondary education programs and activities. It
listed interagency coordination and data matches with, for example,
the Social Security Administration, the Immigration and
Naturalization Service, and the Selective Service as a strategy for
ensuring that postsecondary student aid delivery and program
management are efficient (objective 3.3). Core strategies to achieve
this objective also included working with the Internal Revenue
Service on tax refund offsets and address matches and the Department
of the Treasury on administrative offsets to increase defaulted
student loan collections.
In addition, the Department has taken the important step to revise
the date for its Year 2000 conversion performance indicator. The
plan established 1998, rather than 1999, as the year all of its
relevant computer systems will be Year 2000 compliant, thus allowing
more time for system testing and validation. While the Department
had previously included the Year 2000 conversion effort in its draft
plan, it established in its current plan December 31, 1999, as the
deadline for repairing seven mission critical systems. As we pointed
out, the Year 2000 problem is not technically challenging; however,
it is massive and complex. With about 800 days before the Year 2000
deadline, the current plan's performance indicator of assuring that
all systems have been evaluated and, where necessary, converted to
make them Year 2000 compliant by December 31, 1998, is a major
improvement. In recognition of the critical challenge facing federal
agencies in dealing with this issue, GAO has added the Year 2000
problem as one of its high-risk areas.
OTHER OBSERVATIONS
The Department has made significant strides in its October 1997 plan
in recognizing major management challenges facing the Department. In
our review of its draft strategic plan, we discussed the Department's
particularly difficult challenge in improving its information systems
for the student aid program. We discussed the problems that the lack
of an integrated student financial aid system creates. We also
discussed the Department's reengineering effort, known as "Easy
Access for Students and Institutions (EASI)," which was being
developed to redesign the entire student assistance program delivery
system. In its draft strategic plan, the Department identified EASI
as an important part of its core strategy for integrating its aid
systems. However, as we pointed out, the project had a history of
false starts. We subsequently recommended in another report that the
Department should first develop a systems architecture to address
system integration deficiencies before proceeding with new major
systems development.\1 The Department's plan eliminated EASI from its
core strategies and adopted the broader core strategy of (1)
developing an "integrated, accurate, and efficient student aid
delivery system" and (2) ensuring that systems are mission-driven and
consistent with the Department's information technology architecture.
In our July 18, 1997, report we also highlighted problems with the
Department's management, systems, and processes that affect its
ability to ensure financial accountability, particularly among its
student financial aid programs. The Department recognized these
problems in its strategic plan and listed the following as its most
important challenges: (1) student aid systems that are incompletely
integrated, (2) financial data from aid programs that are only
partially consolidated at the student level, and (3) too many
contractors who use different operating systems. The plan stated
that correcting this situation will require the redesign and
modernization of the federal student financial aid system using the
latest information engineering and computer system technology. To
address these and other issues, the Department included under Goal 3
a new, separate objective for the management of its postsecondary
student financial aid programs: "Postsecondary student aid delivery
and program management is efficient, financially sound, and
customer-responsive" (objective 3.3). The Department identified with
specificity numerous strategies and performance indicators that will
help to address and track agency efforts to achieve this objective.
-- Postsecondary program management. To improve efforts in this
area, the strategic plan states that the Department will develop
and utilize a risk management system to target compliance and
enforcement activities on poorly performing institutions while
reducing burdens on high performing ones. Responding to a
recommendation from its fiscal year 1996 Department-wide
financial audit, the Department is currently developing this new
risk analysis system to better utilize its limited monitoring
resources towards the highest risk institutions. However, this
system will not be fully implemented until fiscal year 1998.
Another Department strategy to improve the management of its
student financial aid programs involves expanding the use of the
case management approach to maximize the effectiveness of
institutional oversight. According to the plan, this approach
encompasses review of recertification applications, compliance
audits, financial statements, risk management system inputs, and
program reviews.
-- Financial integrity. The Department stated in its draft
strategic plan that poor data from the Federal Family Education
Loan Program (FFELP) have prevented it from obtaining an
unqualified audit opinion on its annual financial statements for
the past 4 years. The Department's plan included several core
strategies for addressing this data integrity problem, such as
integrating the multiple student aid databases based on
student-level records and improving contract performance for
major information systems by increased use of performance-based
contracting. The Department also added to its plan an indicator
to track the accuracy and integrity of data supplied by
applicants, institutions, lenders, and guaranty agencies. Data
from these sources have been problematic in the past. In
addition, the Department's October 1997 strategic plan included
a new performance indicator related to the financial integrity
of the Department's postsecondary financial aid programs. It
states that: "There will be no material internal weaknesses
identified in the student aid programs' portions of the
Department-wide financial statement audit and no student aid
program issues that prevent the Department from receiving an
unqualified opinion on the financial statements." This indicator
is linked to and supports indicator 26, which now definitively
states that auditors will issue a clean opinion on the
Department-wide financial statements every year.
Although, in general terms, the plan better specifies how the
Department will address this critical financial management weakness,
it still has not completely clarified how it will resolve the data
integrity issues for FFELP or accurately estimate the government
liability that has prevented the Department from obtaining an
unqualified opinion.
AGENCY COMMENTS
On October 21, 1997, the Department provided written comments on a
draft summary of our observations of its October 3, 1997, strategic
plan. The Department generally had no objections to our observations
but wanted to clarify several issues we raised in the draft.
-- To measure the performance of Department programs, the agency
will include program performance plans in its detailed annual
plan currently being prepared by Department staff in conjunction
with OMB. The individual program plans will be linked directly
to "budget activity lines" and will accompany the Department's
fiscal year 1999 budget justification to Congress in February
1998. The Department submitted 17 draft program plans to
Congress in March 1997 that, among other things, identified each
program's goals and objectives, key performance indicators,
program evaluations and other data sources, the year the
performance indicator data will first be available, and key
strategies for achieving the objectives. These performance
plans were developed by the program offices and have been
reviewed extensively internally--some have been shared with
stakeholders. Program performance plans covering the
Department's approximately 100 program activities will include
essentially the same information as the 17 draft plans and will
be reviewed and updated this fall for inclusion in the agency's
annual plan.
-- The Department's Chief Information Officer has contracted with
Lockheed to work with the agency to ensure that it meets its
Year 2000 performance indicator target of December 31, 1998.
This activity will be monitored at the highest levels within the
agency, and progress will be reported at least quarterly through
the strategic planning tracking process.
-- The Department is engaged in several activities that should help
to resolve the data integrity issues for FFELP and accurately
estimate the government's liability for this program.
Specifically, the Department (1) has developed a workplan,
approved by the independent accounting firm of Price Waterhouse,
to address concerns about the government's liability estimate in
time for the Department's fiscal year 1997 audit; (2) is
comparing data from the National Student Loan Data System
(NSLDS) with audited data submitted by selected guaranty
agencies; and (3) is working with E-Systems, Inc., and direct
loan origination and servicing contractors to ensure the
accuracy and timeliness of direct loan data submitted to NSLDS.
ISSUES AREA CONTACT
Carlotta C. Joyner, Director, Education and Employment Issues;
Health, Education, and Human Services Division, (202) 512-7014.
--------------------
\1 Student Financial Aid Information: Systems Architecture Needed to
Improve Programs' Efficiency (GAO/AIMD-97-122, July 29, 1997)
OBSERVATIONS ON THE DEPARTMENT OF
ENERGY'S STRATEGIC PLAN
========================================================== Appendix VI
On July 11, 1997, we issued a report on the Department of Energy's
(DOE) draft strategic plan dated June 16, 1997 (Results Act:
Observations on the Department of Energy's Draft Strategic Plan,
GAO/RCED-97-199R). DOE formally submitted its strategic plan to OMB
and Congress on September 30, 1997. As requested, we have reviewed
this strategic plan and compared it with the observations in our July
report. On October 14, 1997, we briefed your staffs on our further
observations on DOE's strategic plan. The key points from that
briefing are summarized herein.
SUMMARY OF KEY OBSERVATIONS FROM
OUR REPORTS
As we reported in July 1997, the draft plan did not meet all the
requirements of the Results Act. It fully addressed two of the six
required elements of the Results Act--the mission statement and goals
and objectives--partially addressed a third, and acknowledged that
three others needed to be completed for the September plan.
Furthermore, the draft plan did not expressly link its missions,
goals, objectives, and strategies with DOE's relevant major statutory
responsibilities, although we noted that the missions and activities
defined in DOE's draft plan were generally supported by legislation
and that the draft plan accurately reflected all of DOE's major
legislative requirements. However, we observed that DOE's missions
have evolved from those that Congress envisioned when it created the
Department in 1977 and that the Results Act provides a forum through
which Congress can review the appropriateness of these missions.
Our July 1997 report also noted that the draft plan did not identify
programs and activities that are crosscutting or similar to those of
other federal agencies. In addition, some of the draft plan's
measures addressing management challenges appeared limited in scope
or were unclear. Finally, we noted several weaknesses in the
information system that DOE uses to track performance measures.
In addition to our July report, the Secretary of Energy requested our
continued involvement in refining DOE's plan. On September 2, 1997,
we provided the Department with our comments on its revised draft
strategic plan--dated August 15, 1997 (Results Act: Observations on
the Department of Energy's August 15, 1997, Draft Strategic Plan,
GAO/RCED-97-248R). In that report, we noted that the revised draft
plan was much improved over the earlier draft. Specifically, the
revised plan included all six elements required by the Results Act.
However, we reported that some of the strategies and many of the
measures still did not appear to be results oriented.
IMPROVEMENTS MADE IN DOE'S
STRATEGIC PLAN
DOE's September 30, 1997, strategic plan incorporated several
improvements that make it more responsive to the requirements of the
Results Act than was the June draft plan. In July, we observed that
DOE's draft plan fully addressed two of the six required elements of
the Results Act--the mission statement and goals and
objectives--partially addressed a third, and acknowledged that three
others needed to be completed for the September plan. The September
plan complies with the Results Act requirements by including the
remaining three sections and fully developing the third by adding a
discussion of resource requirements. In describing its resource
requirements, the September plan states that the Department assumed
budget appropriations consistent with the administration's and
Congress' agreed-upon 5-year budget deficit reduction targets through
fiscal year 2002.
Our July report also observed that the draft plan did not expressly
link its missions, goals, objectives, and strategies with DOE's
relevant major statutory responsibilities. The September plan now
shows the linkage between the Department's business line objectives
and its relevant major statutory responsibilities.
Furthermore, DOE's strategic plan now acknowledges--in its discussion
of key external factors--that the Department participates in some
crosscutting government functions and initiatives that are beyond the
mission of any one agency. While the plan does not describe how DOE
will work in concert with other agencies, it does acknowledge DOE's
commitment to work closely with other federal agencies, OMB, and
Congress to ensure that its programs provide critical and unique
contributions to these crosscutting efforts.
DOE'S STRATEGIC PLAN CAN BE
FURTHER IMPROVED
DOE did not adopt all of the suggested improvements noted in our July
and September reports. These suggestions were based on several of
our past reports and represent areas in which we have had
disagreements with DOE in the past. However, we still believe that
if the Department made these suggested changes--as outlined in our
July and September reports--the plan would better address the goals
of the Results Act.
One such example that we identified in our September report relates
to an evaluation that we made concerning the vulnerability of U.S.
oil supplies to disruptions.\1 On the basis of that report, we
believe that DOE's measures for its objective to "reduce the
vulnerability of the U.S. economy to disruptions in energy supplies"
are not very useful indicators of how the Department's programs will
affect the economy's vulnerability. DOE's measures are based on six
strategies: to (1) support activities capable of ending the decline
in domestic oil production, (2) maintain an effective Strategic
Petroleum Reserve, (3) diversify the international supply of oil and
gas, (4) develop alternative transportation fuels and more efficient
vehicles, (5) maximize the productivity of federal oil fields, and
(6) take measures to avoid and respond to domestic energy
disruptions. However, our report on the vulnerability of oil
supplies observed that, in today's world oil market, replacing oil
imports with domestically produced oil would only marginally lower
the potential costs of disruptions because oil prices are set in the
global marketplace and the price for all oil rises during
disruptions.
While we agree that one of DOE's strategies--diversify the
international supplies--can lead to measures that contribute to
reducing the vulnerability of the U.S. economy to disruptions in the
energy supply, our vulnerability report offers five other factors
that we believe would better focus DOE's efforts in developing
strategies and measures for its objective of reducing the
vulnerability to energy supply disruptions: (1) excess world oil
production capacity, (2) the oil intensity of the U.S. economy, (3)
the oil dependency of the U.S. transportation sector, (4) world oil
stocks, and (5) the dependence of the U.S. economy on oil imports.
Finally, our July report noted several weaknesses in the information
system that DOE uses to track performance measures. However, DOE's
September 1997 strategic plan makes no reference to these problems.
We still believe that DOE will need to modify the information system
it anticipates using to track the strategic plan's performance
measures and identify management problems. In addition, we also
noted that the information used to update the tracking system depends
on various other information systems that we and DOE's Inspector
General have found contain incomplete or inaccurate information.
OTHER OBSERVATIONS
While DOE's strategic plan is organized along four business
lines--energy resources, national security, environmental quality,
and science and technology--the agency is organized by program, and
it is not clear from the plan which program offices are accountable
for implementing the different sections of the plan. For example,
several of the Department's program offices have science missions,
including the Office of Nuclear Energy, Science and Technology and
the Office of Energy Research; the Office of Nonproliferation and
National Security and the Assistant Secretary for Defense Programs
have defense missions. However, the plan does not describe how the
Department's current organizational alignment is suited to the plan's
four business lines, nor does it provide a matrix showing which
program offices will be held accountable for implementing each
section of the plan.
AGENCY COMMENTS
On October 10, 1997, we met with DOE officials, including the Acting
Director, Office of Strategic Planning, Budget and Program
Evaluation, to obtain the Department's comments on our observations
about its strategic plan. DOE officials made three points. First,
they stated that development of performance measures is
difficult--especially in the science area--and that they recognize
the need to continually work to improve these measures.
Second, in reference to the disagreements that they had with some of
the policy positions of our past reports, they noted that these
differences will continue; however, they do not believe that they are
strategic planning differences. We disagree because such differences
have an impact on the substance of the plan. For example, if DOE
uses incorrect measures, it will not know if it has achieved its
goals and objectives.
Finally, the officials acknowledged that DOE's strategic plan does
not show program accountability but stated that the Department has
developed a draft matrix document that provides a crosswalk between
its performance measures and the programs. They also pointed out
that after the Department-level matrix is completed, each program
will need to cascade performance measure accountability to it
subunits.
ISSUE AREA CONTACT
Victor S. Rezendes, Director, Energy, Resources, and Sciences
Issues; Resources, Community, and Economic Development Division,
(202) 512-3841.
--------------------
\1 Energy Security: Evaluating U.S. Vulnerability to Oil Supply
Disruptions and Options for Mitigating Their Effects (GAO/RCED-97-6,
Dec. 12, 1996).
OBSERVATIONS ON THE DEPARTMENT OF
HEALTH AND HUMAN SERVICES'
STRATEGIC PLAN
========================================================= Appendix VII
On July 11, 1997, we issued a report on the Department of Health and
Human Services' (HHS) draft strategic plan (The Results Act:
Observations on the Department of Health and Human Services' April
1997 Draft Strategic Plan, GAO/HEHS-97-173R). HHS submitted its
revised strategic plan to the Office of Management and Budget and
Congress on September 30, 1997. As requested, we reviewed the
revised plan and briefed your staffs on our observations. The key
points from that briefing are summarized herein, together with a
brief overview of our comments on the initial HHS plan.
SUMMARY OF KEY OBSERVATIONS FROM
OUR JULY REPORT
We found HHS' draft strategic plan to be missing most of the key
elements required by the Results Act and to be more a summary of
current programs than a document projecting actions HHS might take in
the next several years to achieve the goals of the Act. Although HHS
had developed a mission statement that successfully captured the
broad array of its activities, the draft plan did not define
measurable goals and objectives, describe approaches to achieving
these goals and objectives, describe the relationship between
long-term goals and objectives and annual performance, identify key
external factors beyond HHS' control, or describe how program
evaluations were used to establish or revise strategic goals.
Furthermore, although the draft strategic plan recognized that many
different HHS operating divisions and programs are responsible for
meeting each of HHS' goals, it did not discuss strategies for
coordinating such efforts, nor did it discuss HHS' need