Inspector General Evaluation of the Air Force Systems Engineering Processes Used in the Development of the Refueling Boom for the KC-46A Tanker
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Report No. DODIG-2021-088
I nspec tor Ge ne ral
U.S. Department of Defense
M AY 2 1 , 2 0 2 1
Evaluation of the Air Force
Systems Engineering Processes
Used in the Development of
the Refueling Boom for the
KC-46A Tanker
INTEGRITY INDEPENDENCE EXCELLENCE
Results in Brief
Evaluation of the Air Force Systems Engineering Processes
Used in the Development of the Refueling Boom for the
KC-46A Tanker
May 21, 2021 Finding (cont’d)
Objective testing environment after Boeing officials presented
a system design at the preliminary design review
The objective of this evaluation was in 2012 that differed significantly from the initially
to determine whether the Air Force proposed design; and
adhered to DoD and Air Force systems
• did not verify full functionality of the KC-46A tanker
engineering processes during the design
refueling boom in accordance with the program’s Test
and development of the KC-46A aerial
and Evaluation Master Plan when they performed flight
refueling boom.
testing of the KC-46A tanker refueling boom with
Air Force receiver aircraft.
Background These shortfalls with the KC-46A refueling boom
The KC-46A Pegasus tanker is an aircraft occurred because:
whose mission is aerial refueling of DoD • officials from the KC-46 Program Office did not revalidate
and Allied aircraft. The refueling boom changes to critical technologies or technology maturity
is the component of the aerial refueling at any point during the engineering and manufacturing
system that transfers fuel from the tanker development phase, since revalidations were not
to the receiver aircraft. required by DoD policy; and
In February 2011, the Air Force awarded • officials from the KC-46 Program Office decided,
a fixed-price-incentive contract for the and officials from the Office of the Deputy Assistant
KC-46A tanker to Boeing. Under this Secretary of Defense for Developmental Test and
contract, Boeing is responsible for designing, Evaluation accepted, in 2014 that reduced flight testing
developing, testing, and manufacturing was sufficient to evaluate the performance of the
179 KC-46A tankers for delivery to KC-46A tanker in support of the Milestone C decision
the Air Force. The Air Force contracted in 2016. Despite encountering flight test failures in
with Boeing to deliver the KC-46A tankers January 2016 that required Boeing engineers to redesign
in August 2017; however, deliveries did not the refueling boom, the KC-46 Program Office officials
begin until January 2019. As of October did not change their decision to perform reduced flight
2020, Boeing delivered 38 of the required testing prior to the Milestone C decision. This reduced
179 KC-46A tankers to the Air Force. flight testing did not include the stressing conditions
under which the refueling boom problem could
Finding potentially occur.
KC-46 Program Office officials did not As a result, in 2018, when Boeing attempted to test full
effectively manage the development of functionality of the KC-46A tanker refueling boom after
the refueling boom for the KC-46A tanker. Milestone C, flight test reports documented that refueling
Specifically, KC-46 Program Office officials: boom performance remained a problem during in-flight
refueling of the A-10, C-17, and F-16 receiver aircraft.
• did not ensure that critical
Specifically, the 38 KC-46A tankers that Boeing delivered
technologies for the refueling boom
could not refuel the A-10 or several variants of the
were demonstrated in a relevant
DODIG-2021-088 (Project No. D2019-DEV0SR-0199.000) │ i
Results in Brief
Evaluation of the Air Force Systems Engineering Processes
Used in the Development of the Refueling Boom for the
KC-46A Tanker
Finding (cont’d)
C-130 receiver aircraft, and Air Force officials imposed Additionally, we revised the following
operational limitations allowing the B-52, C-17, F-15, two recommendations:
F-16, F-35A, HC/MC-130J, KC-10, KC-46A, and KC-135 • Use scientific test and analysis techniques to the
receiver aircraft to aerially refuel only under limited maximum extent possible to develop the Test and
flight conditions. Evaluation Master Plan.
In August 2019 and March 2020, the Air Force • Use scientific test and analysis techniques to
issued contract modifications, valued at $100 million, the maximum extent possible to justify the
for the redesign of the KC-46A tanker refueling boom. elimination, deferral, or modification of planned
Had KC-46 Program Office officials effectively managed tests that were originally documented in the Test
the development and testing of the refueling boom for and Evaluation Master Plan.
the KC-46A tanker, the Air Force would not have had to
spend an additional $100 million for the redesign of the Finally, we added two recommendations:
refueling boom to achieve its required performance. • Include the most critical or stressing test conditions
Furthermore, retrofit of the refueling boom for the in the Test and Evaluation Master Plan for any
delivered KC-46A tankers is not estimated to begin until tests where the use of scientific test and analysis
January 2024, and will result in additional undetermined techniques is impractical or not applicable when
costs, as well as approximately a 5-year delayed delivery developing the Test and Evaluation Master Plan.
of the first KC-46A tankers with fully mission-capable • Include the most critical or stressing test
refueling booms. This delay limits the DoD’s use of the conditions in revised test plans when proposing
KC-46A tanker for its intended refueling missions. elimination, deferral, or modification of planned
tests that were originally documented in the Test
Recommendations and Evaluation Master Plan.
Management Comments
We recommend that the Under Secretary of Defense
for Research and Engineering and the Under Secretary
of Defense for Acquisition and Sustainment revise DoD and Our Response
acquisition policy to require program managers of major
The Director of Developmental Test, Evaluation, and
defense acquisition programs to:
Assessments, responding for the Under Secretary
• Conduct knowledge-building technology readiness of Defense for Research and Engineering, in coordination
assessments throughout the acquisition life cycle, with the Under Secretary of Defense for Acquisition
including at preliminary design review, critical and Sustainment, agreed with two recommendations
design review, and Milestone C, at a minimum. related to the use of knowledge-building technology
• Develop and execute technology maturation readiness assessments and technology maturation plans.
plans for critical technologies that have not been Additionally, the Director partially agreed with the
demonstrated in a relevant testing environment, two recommendations related to the use of scientific
as determined by a knowledge-building or test and analysis techniques to develop the Test
statutory technology readiness assessment.
ii │ DODIG-2021-088 (Project No. D2019-DEV0SR-0199.000)
Results in Brief
Evaluation of the Air Force Systems Engineering Processes
Used in the Development of the Refueling Boom for the
KC-46A Tanker
Comments (cont’d)
and Evaluation Master Plan and for proposing the
elimination, modification, or deferral of planned tests
documented in the Test and Evaluation Master Plan.
Although the Director agreed, his comments only
partially addressed our recommendations. Therefore,
the recommendations are unresolved.
We request that the Director provide additional
comments on the final report to describe the specific
actions that the USD(R&E), in coordination with the
USD(A&S), will take to address the recommendations.
Management comments and our response are discussed
in detail in the Recommendations, Management
Comments, and Our Response section of this report.
Please see the Recommendations Table on the next page
for the status of recommendations.
DODIG-2021-088 (Project No. D2019-DEV0SR-0199.000) │ iii
Recommendations Table
Recommendations Recommendations Recommendations
Management Unresolved Resolved Closed
Under Secretary of Defense for Research 1.a, 1.b, 1.c, 1.d,
None None
and Engineering 1.e, 1.f
Under Secretary of Defense for Acquisition 1.a, 1.b, 1.c, 1.d,
None None
and Sustainment 1.e, 1.f
Please provide Management Comments by June 21, 2021.
Note: The following categories are used to describe agency management’s comments to individual recommendations.
• Unresolved – Management has not agreed to implement the recommendation or has not proposed actions that
will address the recommendation.
• Resolved – Management agreed to implement the recommendation or has proposed actions that will address the
underlying finding that generated the recommendation.
• Closed – OIG verified that the agreed upon corrective actions were implemented.
iv │ DODIG-2021-088 (Project No. D2019-DEV0SR-0199.000)
INSPECTOR GENERAL
DEPARTMENT OF DEFENSE
4800 MARK CENTER DRIVE
ALEXANDRIA, VIRGINIA 22350-1500
May 21, 2021
MEMORANDUM FOR UNDER SECRETARY OF DEFENSE FOR RESEARCH AND
ENGINEERING
UNDER SECRETARY OF DEFENSE FOR ACQUISITION AND
SUSTAINMENT
AUDITOR GENERAL, DEPARTMENT OF THE AIR FORCE
SUBJECT: Evaluation of the Air Force Systems Engineering Processes Used in
the Development of the Refueling Boom for the KC-46A Tanker
(Report No. DODIG-2021-088)
This final report provides the results of the DoD Office of Inspector General’s evaluation.
We previously provided copies of the draft report and requested written comments on
the recommendations. We considered management’s comments on the draft report when
preparing the final report. These comments are included in the report.
This report contains recommendations that are unresolved. Comments from the Director
of Developmental Test, Evaluation, and Assessments, responding for the Under Secretary of
Defense for Research and Engineering, in coordination with the Under Secretary of Defense
for Acquisition and Sustainment, partially addressed the recommendations. Additionally, we
made two new recommendations that require comment from the Director.
Therefore, as discussed in the Recommendations, Management Comments, and Our
Response section of this report, the recommendations remain open. We will track these
recommendations until an agreement is reached on the actions that you will take to address
the recommendations, and you have submitted adequate documentation showing that all
agreed-upon actions are completed.
DoD Instruction 7650.03 requires that recommendations be resolved promptly. Therefore,
please provide us within 30 days your response concerning specific actions in process or
alternative corrective actions proposed on the recommendations.
Randolph R. Stone
Assistant Inspector General for Evaluations
Space, Intelligence, Engineering, and Oversight
DODIG-2021-088 │ v
Contents
Introduction
Objective............................................................................................................................................................................................................................ 1
Background.................................................................................................................................................................................................................... 1
Finding. The KC-46 Program Office Did Not
Effectively Manage the Development of the
KC-46A Tanker Refueling Boom........................................................................................................... 9
The KC-46 Program Office Did Not Ensure Critical Technologies for the
Refueling Boom Were Demonstrated in a Relevant Testing Environment. .........................10
DoD Policy Does Not Require Acquisition Programs to Revalidate Technology
Readiness During the Engineering and Manufacturing Development Phase...................14
The KC-46 Program Office Did Not Verify Full Functionality of the Refueling
Boom During Flight Testing. ....................................................................................................................................................... 18
The KC-46 Program Office Reduced the Scope of Flight Testing for Initial
Refueling Boom Verification....................................................................................................................................................... 23
The Air Force Issued $100 Million in Contract Modifications for the Redesign
of the Refueling Boom With Additional Schedule Impacts Expected........................................ 24
Recommendations, Management Comments, and Our Response. ............................................................ 25
Appendixes
Appendix A. Scope and Methodology. ....................................................................................................................................... 30
Use of Computer-Processed Data.......................................................................................................................................... 32
Prior Coverage............................................................................................................................................................................................. 32
Appendix B. Timeline of Major Events in the KC-46A Tanker Modernization
Program Related to the Refueling Boom Problems. ....................................................................................... 34
Management Comments................................................................................................................................... 35
Under Secretary of Defense for Research and Engineering and the Under
Secretary of Defense for Acquisition and Sustainment.............................................................................. 35
Acronyms and Abbreviations. ............................................................................................................... 38
vi │ DODIG-2021-088
Introduction
Introduction
Objective
The objective of this evaluation was to determine whether the Air Force adhered
to DoD and Air Force systems engineering processes during the design and
development of the KC-46A tanker refueling boom.1 Specifically, we determined
whether the systems engineering processes for the KC-46A Tanker Modernization
program led to a critical deficiency with the KC-46A tanker aerial refueling
capability that the Air Force encountered during post-Milestone C flight testing.
Background
Defense Acquisition System Milestones
DoD Directive 5000.01 establishes the policies and principles that govern the
defense acquisition system, and forms the management foundation for all DoD
acquisition programs. 2 The defense acquisition system’s primary objective is
to acquire quality products and satisfy user needs with measurable improvement
in mission. The defense acquisition system’s major capability acquisition process
includes three DoD program decision points to assess a program’s readiness to
proceed to the next acquisition phase—Milestone A, Milestone B, and Milestone C.
The milestone decision authority decides whether the program is ready to transition
to the next acquisition phase and is the sole and final decision authority. 3
At the Milestone A decision, the milestone decision authority decides whether
an acquisition program can enter the technology maturation and risk reduction
phase, where program officials reduce the weapon systems risks, develop capability
requirements, and finalize affordability caps. At the Milestone A review,
program officials present program documentation, such as their acquisition
strategy, program risk assessment, and affordability analysis, to the milestone
decision authority.
At the Milestone B decision, the milestone decision authority decides whether
an acquisition program can enter the engineering and manufacturing development
phase where program officials develop, build, and test that the weapon system
1
Air Force Instruction 63-1201, “Life Cycle Systems Engineering,” defines systems engineering as an interdisciplinary
approach encompassing the entire set of scientific, technical, and managerial efforts needed to evolve, verify, deploy,
and support an integrated and life-cycle-balanced set of system solutions that satisfy customer needs. The refueling
boom is the component of the aerial refueling system that transfers fuel from the KC-46A tanker to the receiver aircraft.
2
DoD Directive 5000.01, “The Defense Acquisition System,” September 9, 2020.
3
The milestone decision authority for the KC-46A tanker program was the Under Secretary of Defense for Acquisition,
Testing and Logistics until November 2017, when the Under Secretary of Defense for Acquisition and Sustainment
delegated the milestone decision authority to the Secretary of the Air Force.
DODIG-2021-088 │ 1
Introduction
meets all requirements to support production or deployment decisions. At the
Milestone B review, program officials present program documentation to the
milestone decision authority, such as their desired capability requirements,
an independent cost estimate, and the acquisition program baseline.
At the Milestone C decision, the milestone decision authority decides whether
an acquisition program can enter the production and deployment phase where
program officials produce and deliver requirements-compliant products to
receiving military organizations. At the Milestone C review, program officials
present program documentation to the milestone decision authority to demonstrate
that the weapon system is stable, within the approved affordability caps, and will
meet requirements.
Developmental Test and Evaluation
Developmental test and evaluation is conducted throughout the acquisition
process to assist in engineering design and development and to verify that
technical performance specifications have been met. The Test and Evaluation
Master Plan (TEMP) serves as the primary document for managing a test and
evaluation program. The TEMP contains an integrated test program summary
and master schedule of all major test events or test phases. Program officials
update the TEMP as needed to support acquisition milestones and decision points.
The program manager uses the TEMP as the planning and management tool for
all program test activities.
KC-46A Tanker Modernization Program
The KC-46A Tanker Modernization program is a major defense acquisition
program. 4 In February 2011, the Air Force awarded a contract to The Boeing
Company (Boeing) to develop an aerial refueling tanker by modifying a commercial
767 passenger aircraft and designated this modified aircraft the KC-46A Pegasus
tanker, shown in Figure 1. The KC-46A Pegasus tanker is an aircraft whose mission
is aerial refueling of DoD and allied aircraft. Boeing is currently producing the
KC-46A tanker for the Air Force to replace its aging fleet of KC-135 tanker aircraft
that have been a significant component of the DoD’s refueling aircraft fleet since
approximately 1956.
An integral part of the KC-46A modernization program is the refueling boom.
Figure 1 illustrates the location of the refueling boom at the rear of the
KC-46A tanker. Figure 2 illustrates the components that make up the refueling
4
Section 2430, title 10, United States Code, 2012, defines a major defense acquisition program as a DoD acquisition
program that is either designated by the Secretary of Defense as a major defense acquisition program or that is
estimated for all increments of the program to require an eventual total expenditure for research, development,
and test and evaluation of more than $525 million in fiscal year 2020 constant dollars or, for procurement, of more
than $3.065 billion in fiscal year 2020 constant dollars.
2 │ DODIG-2021-088Introduction
boom, including the pivot point where the boom mounts to the KC-46A tanker, the
telescoping section that moves in and out to accommodate movement of receiver
aircraft during refueling operations, and the nozzle that transfers fuel to receiver
aircraft. In addition to the refueling boom, the KC-46A tanker is equipped with
a centerline drogue refueling system and wing aerial refueling pods which provide
expanded aerial refueling capabilities. 5
Figure 1. KC-46A Tanker Key Features
Source: The Air Force.
5
A drogue refueling system consists of a funnel-shaped device that is attached to the end of a long flexible hose
suspended from a tanker aircraft in flight and into which the probe of a receiver aircraft connects to receive fuel
from the tanker.
DODIG-2021-088 │ 3Introduction
Figure 2. KC-46A Tanker Refueling Boom
Source: The Air Force.
The refueling boom is the component of the aerial refueling system that transfers
fuel from the KC-46A tanker to the receiver aircraft. A receiver aircraft is any
DoD or allied aircraft capable of being aerially refueled. During aerial refueling,
an operator on the KC-46A tanker extends the refueling boom into a receptacle
on the receiving aircraft to transfer fuel. The goal of aerial refueling is to extend
the range or time a receiver aircraft can remain in the air.
The Air Force Initiated the KC-46A Tanker Modernization
Program in 2011
In February 2011, KC-46 Program Office officials initiated the KC-46A Tanker
Modernization program at Milestone B and entered the engineering and manufacturing
development phase of the DoD acquisition process. Shortly after program initiation
in 2011, Air Force officials awarded Boeing a fixed-price-incentive contract
for the acquisition of 4 KC-46A tankers. Including options to procure up to an
additional 175 KC-46A tankers, for a total of 179 aircraft, this contract was valued
at $41.5 billion. The engineering and manufacturing development phase of the
KC-46A Tanker Modernization program ended in August 2016, upon Milestone C
approval. The Air Force contract required Boeing to deliver the KC-46A tankers in
August 2017; however, deliveries did not begin until January 2019. As of October 2020,
Boeing had delivered 38 of the 179 KC-46A tankers to the Air Force.
4 │ DODIG-2021-088Introduction
KC-46A Tanker Refueling Boom Performance
During developmental testing prior to Milestone C in early 2016, Air Force pilots
reported performance problems to KC-46 Program Office officials regarding
the operation of the KC-46A tanker refueling boom with C-17 receiver aircraft. 6
Specifically, the boom axial loads were too high during testing, meaning that the
boom was too stiff and would not extend or retract during flight testing unless
subjected to more force than the system performance specification required.7
Initially, Boeing engineers implemented software updates to the refueling boom
control system in an attempt to optimize the performance of the refueling boom,
and then decided to pursue a software and hardware solution to reduce refueling
boom axial forces. However, after Boeing engineers made software and hardware
updates, additional flight tests in 2018 demonstrated that refueling boom
performance remained a problem due to high axial loads during in-flight refueling
of the A-10, C-17, and F-16 receiver aircraft.
The refueling boom performance problems caused Air Force Materiel Command
flight test officials to issue a Category I performance deficiency in September 2018,
identifying the boom as being too stiff while in contact with the receiver aircraft. 8
The performance deficiency report stated that excessive receiver aircraft engine
thrust was necessary to compress the boom under certain aerial refueling
conditions. Additionally, the deficiency report documented that receiver aircraft
needed to make large engine power corrections when attempting to make a
forward or backward position adjustment to maintain contact with the refueling
boom. The large engine power corrections could result in potentially unsafe
flight operations during the process of disconnecting the receiver aircraft from
the refueling boom. Because the refueling boom was too stiff, it caused pilots
of receiver aircraft to inadvertently use excess engine power or not use enough
engine power, which, upon disconnecting from the refueling boom, could cause
the receiver aircraft to rapidly accelerate toward or away from the tanker.
The rapid aircraft acceleration could cause the receiver aircraft to lunge into
the refueling boom, potentially causing damage to the receiver aircraft, the
refueling boom, or both.
6
Developmental test and evaluation is conducted throughout the acquisition process to assist in engineering design
and development and to verify that technical performance specifications have been met.
7
Axial load is the force applied on a structure directly along an axis.
8
According to Air Force Instruction 99-103, “Capabilities-Based Test and Evaluation,” Category I deficiencies are those
that may cause death, severe injury, or severe occupational illness; may cause loss or major damage to a weapon
system; critically restrict the combat readiness capabilities of the using organization; or result in a production line
stoppage. Boeing engineers designed the KC-46A tanker refueling boom to extend and retract, adjusting the length
of the boom so that contact with the receiver aircraft can be maintained for aerial refueling operations.
DODIG-2021-088 │ 5Introduction
Subsequently, in December 2018, KC-46 Program Office officials established new
technical requirements for the redesign of the KC-46A tanker refueling boom and
issued associated contract modifications in 2019 and 2020. As of October 2020,
Boeing had delivered 38 of the 179 KC-46A tankers to the Air Force. However,
the 38 KC-46A tankers that were delivered did not have the redesigned refueling
boom and, as a result, none of the delivered tankers could refuel the A-10 or several
variants of the C-130 receiver aircraft.9 To address the refueling boom deficiencies,
the Air Force imposed operational limitations on these 38 tankers allowing the
B-52, C-17, F-15, F-16, F-35A, HC/MC-130J, KC-10, KC-46A, and KC-135 receiver
aircraft to be aerially refueled only under limited flight conditions. Examples
of limited flight conditions included reduced tanker refueling boom range of
motion and no refueling in a covert or lights-out scenario.
KC-46A Tanker Modernization Program Stakeholders
There are several DoD and Air Force organizations that have roles and
responsibilities for the KC-46A program.
Air Force Life Cycle Management Center
The Air Force Life Cycle Management Center (AFLCMC) is located at Wright-Patterson
Air Force Base in Dayton, Ohio. The AFLCMC is responsible for life cycle management
of Air Force weapon systems from inception to retirement. Ten Program Executive
Officers within AFLCMC, including the Program Executive Officer for the Mobility
and Training Aircraft Directorate, are responsible for the activities within
their respective portfolio and report to the Assistant Secretary of the Air Force
for Acquisition.10
KC-46 Program Office
The KC-46 Division of the AFLCMC Mobility and Training Aircraft Directorate
(KC-46 Program Office) is responsible for the planning and execution of all
life-cycle activities for the Air Force’s KC-46A tanker fleet. The KC-46 Program
Office is located at Wright-Patterson Air Force Base, Ohio. The life-cycle activities
that the KC-46 Program Office is responsible for include the development, test,
production, fielding, and support of the KC-46A tanker.
9
The variants of the C-130 aircraft that the KC-46A tanker could not refuel included the AC-130J, AC-130W, EC-130J,
MC-130H, and MC-130J.
10
In May 2020, the Air Force reorganized the AFLCMC, realigning three tanker program offices from the Tanker Directorate
to the Mobility and Training Aircraft Directorate.
6 │ DODIG-2021-088Introduction
Deputy Assistant Secretary of Defense for Developmental Test
and Evaluation
Prior to June 2018, the Deputy Assistant Secretary of Defense for Developmental
Test and Evaluation (DASD[DT&E]) was the principal advisor to the then-Secretary
of Defense and the Under Secretary of Defense for Acquisition, Technology, and
Logistics (USD[AT&L]).11 The DASD(DT&E)’s primary responsibilities included
establishing policy and guidance for the conduct of developmental test and
evaluation in the DoD and assessing the developmental test and evaluation
activities of each major defense acquisition program.
DoD Revised Acquisition Policy During the KC-46A Tanker
Modernization Program
During the engineering and manufacturing development phase of the KC-46A Tanker
Modernization program, the DoD revised DoD Instruction (DoDI) 5000.02 twice—once
in November 2013 and again in January 2015.12 The version of DoDI 5000.02 issued
in December 2008, in effect at Milestone B of the KC-46A Tanker Modernization
program, required Acquisition Category ID programs—such as the KC-46A Tanker
Modernization program—to conduct a technology readiness assessment (TRA) at
both Milestone B and Milestone C.13 A TRA is a formal metrics-based process and
accompanying report that assesses the maturity of critical hardware and software
technologies called critical technology elements. In November 2013, DoDI 5000.02
was revised to require all major defense acquisition programs to conduct a TRA
before release of development contract requests for proposal, with updates to the
TRA before Milestone B if changes had occurred, and a TRA at Milestone C only
if the program was initiated at Milestone C. DoDI 5000.02 was revised again in
January 2015; however, the requirements related to TRAs remained the same as
in the version of DoDI 5000.02 issued in 2013.
11
As of June 25, 2018, in accordance with Public Law 114-328, the “National Defense Authorization Act for Fiscal
Year 2017,” Section 901, “Organization of the Office of the Secretary of Defense,” the DoD reorganized the Office of
the USD(AT&L) to establish the Office of the Under Secretary of Defense for Research and Engineering and the Office
of the Under Secretary of Defense for Acquisition and Sustainment. Additionally, the DoD reorganized the Office of the
DASD(DT&E) to establish the Office of the Deputy Director for Developmental Test, Evaluation, and Prototyping within
the Office of the Under Secretary of Defense for Research and Engineering.
12
DoDI 5000.02, “Operation of the Defense Acquisition System,” December 8, 2008. DoDI 5000.02, “Operation of the
Defense Acquisition System,” November 26, 2013. DoDI 5000.02, “Operation of the Defense Acquisition System,”
January 7, 2015.
13
An Acquisition Category ID program is a major defense acquisition program, as defined in Section 2430, title 10,
United States Code, for which the Under Secretary of Defense for Acquisition and Sustainment makes a decision
to become the milestone decision authority or designate another Office of the Secretary of Defense official as the
milestone decision authority. The milestone decision authority is the designated individual with overall responsibility
for a program and with the authority to approve entry of an acquisition program into the next phase of the
acquisition process.
DODIG-2021-088 │ 7Introduction
In January 2020, the DoD further restructured the acquisition process, referred
to as the Adaptive Acquisition Framework, which established six acquisition
“pathways,” including the major capability acquisition pathway.14 Accordingly,
in August 2020, the DoD issued DoDI 5000.85 to establish policy and prescribe
procedures that guide the acquisition of major capability acquisition programs,
including major defense acquisition programs.15 Additionally, in November 2020,
the DoD issued DoDI 5000.88 to establish policy, assign responsibilities, and
provide procedures for the engineering management activities necessary to guide
the development of defense systems.16
14
DoDI 5000.02, “Operation of the Adaptive Acquisition Framework,” January 23, 2020.
15
DoDI 5000.85, “Major Capability Acquisition,” August 6, 2020.
16
DoDI 5000.88, “Engineering of Defense Systems,” November 18, 2020.
8 │ DODIG-2021-088Finding
Finding
The KC-46 Program Office Did Not Effectively Manage
the Development of the KC-46A Tanker Refueling Boom
KC-46 Program Office officials did not effectively manage the development of the
refueling boom for the KC-46A tanker. Specifically, KC-46 Program Office officials:
• did not ensure that critical technologies for the refueling boom were
demonstrated in a relevant testing environment after Boeing officials
presented a system design at the preliminary design review in 2012
that differed significantly from the initially proposed design; and
• did not verify full functionality of the KC-46A tanker refueling boom
in accordance with the program’s TEMP when the Air Force performed
flight testing of the KC-46A tanker refueling boom with Air Force receiver
aircraft, including the A-10, C-17, and F-16, in 2016.
These shortfalls with the KC-46A refueling boom occurred because:
• officials from the KC-46 Program Office did not revalidate changes
to critical technologies or technology maturity during the engineering
and manufacturing development phase since revalidations were not
required by any version of DoDI 5000.02; and
• officials from the KC-46 Program Office decided, and officials from
the Office of the DASD(DT&E) accepted, in 2014 that reduced flight
testing was sufficient to evaluate the performance of the KC-46A tanker
in support of the Milestone C decision in 2016. Despite encountering
KC-46A flight test failures in January 2016 that required Boeing engineers
to redesign the refueling boom, the KC-46 Program Office officials did
not change their decision to perform reduced flight testing prior to the
Milestone C decision. This reduced flight testing did not include the
stressing conditions under which the refueling boom problem could
potentially occur.
As a result, despite actions by Boeing to correct the refueling boom performance
problems, when Boeing attempted to test full functionality of the KC-46A tanker
refueling boom after Milestone C in 2018, flight test reports documented that
refueling boom performance remained a problem during in-flight refueling of the
A-10, C-17, and F-16 receiver aircraft. As of October 2020, Boeing had delivered
38 of 179 total KC-46A tankers to the Air Force, however, the 38 KC-46A tankers
that Boeing delivered could not refuel the A-10 or several variants of the C-130
receiver aircraft. In addition, the DoD continued to experience problems when
refueling the B-52, C-17, F-15, F-16, F-35A, HC/MC-130J, KC-10, KC-46A, and
KC-135 receiver aircraft, which the Air Force addressed by limiting the flight
conditions for aerial refueling operations with these receiver aircraft.
DODIG-2021-088 │ 9Finding
The Air Force issued contract modifications in August 2019 and March 2020,
valued at $100 million, for the redesign of the KC-46A tanker refueling
boom. These modifications did not include the acquisition of the redesigned
refueling booms and their installation on the KC-46A aircraft already delivered.
The associated contract modifications will be negotiated at a later time. According
to the KC-46A program officials, the Air Force plans to start the installation of
the redesigned refueling booms on the delivered KC-46A aircraft in January 2024.
Had KC-46 Program Office officials effectively managed the development and
testing of the refueling boom for the KC-46A tanker, the Air Force would not
have had to spend an additional $100 million for the redesign of the refueling
boom to achieve its required performance. Furthermore, retrofit of the
refueling boom for the delivered KC-46A tankers is not estimated to begin
until January 2024, and will result in a delay of approximately 5 years for the
delivery of the first KC-46A tankers with full mission-capable refueling booms.
This delay limits the DoD’s use of the KC-46A tanker for its intended refueling
missions. Additionally, the Commander of United States Transportation Command
identified the aerial refueling fleet as the most stressed of the air mobility forces,
and stated that any delay of the KC-46 production puts the Joint Force’s ability
to effectively execute war plans at risk. Finally, Congress included minimum
inventory requirements and prohibitions on the retirement of legacy KC-10 and
KC-135 refueling tankers in the FY 2021 National Defense Authorization Act.
The KC-46 Program Office Did Not Ensure Critical
Technologies for the Refueling Boom Were
Demonstrated in a Relevant Testing Environment
The KC-46 Program Office did not ensure that critical technologies for the
KC-46A tanker refueling boom were demonstrated in a relevant testing
environment after Boeing officials presented a system design at the preliminary
design review in 2012 that differed significantly from the proposed design at the
time of the TRA for the Milestone B decision in 2011.17 Section 2366b, title 10,
United States Code, 2010, requires the milestone decision authority to certify, based
on an independent review, that technology in a major defense acquisition program
has been “demonstrated in a relevant environment” prior to Milestone B approval.
The DoD defines a relevant environment as “a testing environment that simulates
both the most important and most stressing aspects of the operational environment.”18
17
Preliminary design review is a technical assessment to ensure the preliminary design and basic system architecture are
complete, that there is technical confidence the capability need can be satisfied within cost and schedule goals, and that
risks have been identified and mitigation plans established.
18
The DoD Technology Readiness Assessment (TRA) Deskbook, published in 2009, provides definitions of terms associated
with technology maturity, including “relevant environment” and “critical technology element.”
10 │ DODIG-2021-088Finding
In accordance with the DoDI 5000.02 that was in effect during the KC-46A Tanker
Modernization program Milestone B decision in 2011, technology procured from
industry “shall have been demonstrated in a relevant environment or, preferably,
in an operational environment to be considered mature enough to use for product
development.” The DoDI 5000.02 in effect at Milestone B required a program
office to conduct a TRA for major defense acquisition programs and provide the
results to the milestone decision authority at both Milestone B and Milestone C.
However, prior to the KC-46A Tanker Modernization program Milestone C decision
in August 2016, DoDI 5000.02 was revised to require major defense acquisition
programs to conduct a TRA only at the milestone of program initiation in the DoD
acquisition life cycle. This policy change eliminated the requirement to conduct
a TRA for the KC-46A Tanker Modernization program at Milestone C.
According to DoD TRA guidance, a program office will establish an independent
review team of subject matter experts to conduct a TRA. A TRA is a formal
metrics-based process and accompanying report that assesses the maturity of critical
hardware and software technologies called critical technology elements. A TRA
independent review team examines program concepts, technology requirements,
and demonstrated capabilities. For the purposes of a TRA, a technology element is
critical if “the system being acquired depends on this technology element to meet
operational requirements (within acceptable cost and schedule limits) and if the
technology element or its application is either new or novel or in an area that poses
major technological risk during detailed design or demonstration.”19
The KC-46 Program Office Conducted a Technology Readiness
Assessment and Found the Refueling Boom Was Not a Critical
Technology in 2011
In support of the Milestone B decision, the KC-46 Program Office formed an
independent review team to conduct a TRA focused on technology readiness of the
KC-46A Tanker Modernization Program, which determined that the KC-46A tanker
refueling boom was not a critical technology element. The Assistant Secretary
of Defense for Research and Engineering approved the TRA report in February 2011.
The TRA report stated that the independent review team determined that Boeing
engineers used no new or novel technology in the design of the KC-46A tanker
refueling boom because the design was “based on that of the well-proven KC-10
[refueling boom] and the control laws [were] based on the Italian KC-767A and
Japanese KC-767J control laws.”20 As a result, the independent review team
completed no further assessment to ensure that the refueling boom technologies
had been demonstrated in a relevant testing environment.
19
The DoD Technology Readiness Assessment Deskbook, 2009.
20
Refueling boom control laws are the mathematical rules that control the refueling boom’s flight control surfaces and
extension of the boom based on the aerial refueling operator’s control inputs, so that the boom can be maneuvered
throughout its operational envelope (its range of allowable positions and orientations).
DODIG-2021-088 │ 11Finding
According to the TRA report, the independent review team identified the
critical technology elements used in the KC-46A tanker based on the results
of Boeing’s assessment of the technology maturity and information in Boeing’s
proposal, including:
• mission capability and non-mandatory technical requirements,
• design approach description,
• system development and management plan, and
• contract work breakdown structure.
The TRA report states that when Boeing’s proposal lacked enough information
to determine whether a technology should be identified as a critical technology
element, the independent review team requested additional information from
Boeing, and in some cases, the information from Boeing was supplemented with
outside research (including technical reports, technical papers, and interviews
with subject matter experts).
However, the KC-46 Program Office was unable to provide any documentation
that supported the independent review team’s determination that the KC-46A
tanker refueling boom was not a critical technology element. Therefore, we
were unable to independently verify the TRA review team’s determination that
the KC-46A tanker refueling boom should not have been identified as a critical
technology element.
Boeing Officials Presented a Refueling Boom Design to the
Air Force in February 2012 that Differed Significantly from
the Design in the Technology Readiness Assessment Report
in 2011
After the Air Force completed the TRA in 2011, Boeing officials presented refueling
boom design details to the Air Force during the preliminary design review in
February 2012. We reviewed the preliminary design review documentation and
found that it showed a refueling boom design that differed significantly from the
proposed design that the independent review team documented in the TRA report.
Based on our analysis of the preliminary design review documentation, we found
that the KC-46A tanker refueling boom included a computer control system that
was an integral part of its function. In contrast, the mature technology of the
KC-10 refueling boom—which the KC-46A tanker refueling boom was proposed
to be based upon—did not include a computer control system.
12 │ DODIG-2021-088Finding
According to Air Force aerial refueling engineers, the KC-10 refueling boom was
a hydromechanically controlled system. The KC-10 refueling boom design did not
include the measurement of boom axial loads and use of a computer system to
actively control refueling boom axial loads. However, the KC-46A refueling boom
used sensors and a computer to measure refueling boom axial loads and to process
the measurements to actively control movement of the refueling boom. Based on
this information, we concluded that the KC-46A refueling boom included new or
novel technology and, therefore, should have been identified as a critical technology
element. 21 Therefore, KC-46 Program Office officials should have ensured that
the refueling boom was demonstrated in a relevant testing environment before
proceeding beyond Milestone B.
KC-46 Program Office Officials Did Not Revalidate the TRA
When Presented with Information Showing KC-46A Tanker
Refueling Boom Technologies Were New or Novel
We found that the KC-46 Program Office did not revalidate the technology
readiness assessment of the KC-46A tanker refueling boom after being presented
with new technical information during the preliminary design review. Air Force
aerial refueling engineers for the KC-46A tanker stated that their first indication
the KC-46A tanker refueling boom technology was not based on the KC-10 refueling
boom design was during the Milestone C decision in 2016. However, e-mail
correspondence between AFLCMC aerial refueling engineers and KC-46 Program
Office officials in 2013—before the Milestone C decision in 2016—showed that
Air Force aerial refueling engineers were aware that the KC-46A tanker refueling
boom relied on a computer control system to function.
AFLCMC aerial refueling engineers and KC-46 Program Office officials told us in
2020 that they would have assessed the KC-46A tanker refueling boom technology
to be new or novel if they had known the operation of the refueling boom was
computer controlled and not hydromechanically controlled, as in the KC-10.
21
According to the DoD Technology Readiness Assessment Deskbook, 2009, a critical technology element is a technology
element that (1) the system being acquired depends on to meet operational requirements and (2) is either new or novel
in application or in an area that poses major technological risk during detailed design or demonstration.
DODIG-2021-088 │ 13Finding
DoD Policy Does Not Require Acquisition Programs
to Revalidate Technology Readiness During the
Engineering and Manufacturing Development Phase
KC-46 Program Office officials did not revalidate changes to critical technologies
or technology maturity during the engineering and manufacturing development
phase because revalidations were not, and still are not, required by any version
of DoDI 5000.02. 22
Knowledge-Building Technology Readiness Assessments Are
Necessary Throughout System Development
Performing continuous knowledge-building TRAs throughout development could
benefit acquisition programs by mitigating the cost and schedule overruns these
programs have experienced due to lack of technology maturity. 23 The Government
Accountability Office (GAO) best practices recommend reviewing technology
maturity as needed throughout the product development life cycle. Additionally,
GAO best practices state that TRAs are important inputs into systems engineering
events, such as a project’s preliminary design review and critical design review,
and can expose knowledge gaps.
However, the DoD does not require acquisition programs to conduct a TRA once
programs have proceeded beyond Milestone B, unless Milestone C is the point
of initiation of the program. If DoD acquisition policy required major defense
acquisition programs to revalidate critical technologies and technology maturity
throughout the engineering and manufacturing development phase, then KC-46
Program Office officials may have identified the KC-46A tanker refueling boom
as a critical technology that needed to be further matured before the program
entered production.
According to the GAO’s TRA guide, “the expert community has recognized that
more frequent, regular assessments of the maturity of a project’s or program’s
critical technologies are also best practices….” The GAO states, “some experts
have been concerned that applying the same set of practices to these more
22
DoDI 5000.02, “Operation of the Defense Acquisition System,” December 8, 2008. During the engineering and
manufacturing development phase of the KC-46A Tanker Modernization program, the DoD revised DoDI 5000.02
multiple times, including in 2013 and 2015. Ultimately, the DoD restructured the acquisition process in January 2020
and issued DoDI 5000.02, “Operation of the Adaptive Acquisition Framework,” January 23, 2020. DoDI 5000.02 still
does not require program offices to revalidate changes to critical technologies or technology maturity for major defense
acquisition programs during the engineering and manufacturing development phase.
23
GAO Report No. GAO-16-410G, “Technology Readiness Assessment Guide: Best Practices for Evaluating the Readiness
of Technology for Use in Acquisition Programs and Projects,” August 2016—subsequently updated and reissued as GAO
Report No. GAO-20-48G in January 2020. The GAO developed this TRA guide to: (1) describe generally accepted best
practices for conducting effective evaluations of technology developed for systems or acquisition programs; and
(2) provide program managers, technology developers, and governance bodies with the tools they need to more
effectively mature technology, determine its readiness, and manage and mitigate risk.
14 │ DODIG-2021-088Finding
frequent assessments might make them too time consuming and cost prohibitive
and ultimately dissuade technology and program managers from conducting
them.” However, the GAO states that these program self-assessments (referred to
as knowledge-building technology readiness assessments) could be conducted by or
for a narrow audience—for instance, the program manager or systems engineers—
to assess the progress in achieving technical maturity for a specific technology or
group of technologies. Additionally, the GAO states that organizations conducting
tailored TRAs in periods between decision points as knowledge-building exercises
can put program managers and technology developers in a better position to gauge
progress, monitor and manage technology maturity, and identify and manage risks
before they become more costly.
Immature Technologies Are a Systemic Cause of Problems in
DoD Acquisition Programs
According to the GAO’s TRA guide, the GAO found that, in many DoD acquisition
programs, cost growth and schedule delays resulted from overly optimistic
assumptions about technology maturity. Additionally, the GAO stated that during
product development, TRAs are important inputs into systems engineering
events, such as a program’s critical design review, and can expose knowledge
gaps. If a system’s critical technologies are not mature and have not at least been
demonstrated in a relevant environment at the point of the program’s critical
design review, then the program may not have a solid technical basis on which
to develop its design.
According to several prior DoD OIG and GAO reports, technology maturity problems
occurred in other DoD programs. The following examples are of DoD acquisition
program failures where program offices used immature technologies and failed
to demonstrate those immature technologies in a relevant environment.
Ford Class Aircraft Carrier Advanced Arresting Gear
DoD OIG Report No. DODIG-2016-107 found that ten years after the program
entered the engineering and manufacturing development phase, the Navy was not
able to prove the capability or safety of the system to a level that would permit
actual testing of the system on an aircraft carrier because of hardware failures
and software challenges. This occurred because the Navy pursued a technological
solution for its Ford-class carriers that was not sufficiently mature for the planned
use, resulting in hardware failures to mechanical and electrical components, and
software modifications to accommodate these failures. 24
24
DODIG-2016-107, “Advanced Arresting Gear Program Exceeded Cost and Schedule Baselines,” July 5, 2016.
DODIG-2021-088 │ 15Finding
As a result, the advanced arresting gear system required significant redesign
of mechanical and electrical components to meet system requirements, which
delayed developmental testing and contributed to research, development,
test, and evaluation costs increasing $571.5 million from the 2005 Acquisition
Program Baseline.
National Polar-Orbiting Operational Environmental
Satellite System
GAO Report No. GAO-07-96 found that costs for DoD space acquisitions have been
consistently underestimated over the past several decades, sometimes by billions
of dollars. In 2006, the GAO showed that cost growth in DoD space programs was
largely caused by initiating programs before determining whether requirements
were achievable. On the National Polar-orbiting Operational Environmental
Satellite System program, only 1 of 14 critical technologies was mature at program
initiation, and 1 technology was found to be less mature after the contractor
conducted more verification testing. The combination of optimistic cost estimates
with immature technology resulted in cost increases and schedule delays.
U.S. Army Crusader
GAO Report No. GAO-02-201 found that the Army identified requirements for
the Crusader, a lighter and more deployable advanced field artillery system,
to replace the Paladin system in 2000. In 2002, the GAO found that the Army
had overestimated the maturity of critical technologies and risked cost overruns,
schedule delays, and performance shortfalls by prematurely committing the
program to product development. The GAO stated that the DoD viewed the Army’s
long time experience with certain technologies within the program as one reason
for the Army’s failure to identify all critical technologies. The GAO recommended
that the Army further mature the Crusader’s technologies before committing
to product development and assess the projected capabilities and fielding schedules
for future combat systems as part of the Crusader’s milestone decision for
beginning product development.
Columbia Class Submarine
GAO Report No. GAO-18-158 found that the Navy conducted the 2015 Columbia
class TRA in accordance with a DoD-approved plan; however, it did not follow
best practices for identifying all critical technology elements, resulting in an
underrepresentation of the technical risk facing the program. Specifically, the
TRA only identified two critical technology elements, but the GAO identified
four additional critical technology elements. The GAO determined that the
Columbia class TRA did not identify all appropriate critical technology elements
16 │ DODIG-2021-088Finding
because the Navy used a more restrictive definition of a critical technology element
than that recommended in the GAO best practices guide. The GAO guide states that
reused existing technologies can be critical if they are used in a different form, fit,
or function. The GAO concluded that, given the risks facing the program and the
significance of potential delays or cost growth, this program warranted increased
scrutiny of its critical technologies, several of which were immature.
Technology Maturation Plans Are Necessary for Critical
Technologies That Are Less Mature Than Required
According to the GAO’s TRA guide, it is important for program managers to develop
a plan for maturing critical technologies that are rated at a lower technology
readiness level than expected. After completion of the TRA, program managers
should provide the appropriate information for maturing new technology in other
key planning and analytical documents, such as technology maturation plans and
risk management plans.
Technology maturation plans establish a road map for maturing technologies to
a designated or higher technology readiness level. According to the GAO’s TRA
guide, a technology maturation plan is a “management planning tool that lays
out the steps, actions, and resources needed for maturing critical technologies
that have been assessed as less mature than desired or are lagging in maturity
compared to other critical technologies.”
Technology maturation plans are also useful as key reference documents at
program milestones to verify that a program has made adequate progress in
closing the maturity gaps. Programs should update technology maturation plans
as progress is made, new information is discovered, or as conditions that materially
affect the plans occur. If DoD acquisition policy required major defense acquisition
programs to use technology maturation plans to ensure that critical technologies
are matured to the required technology readiness levels, then KC-46 Program
Office officials may have taken additional action to develop the KC-46A tanker
refueling boom critical technologies before entering production—potentially
minimizing the costly redesign efforts and delayed delivery of the
modernized tankers.
DODIG-2021-088 │ 17You can also read
