SYNTHESIZING SOS CONCEPTS FOR USE IN COST MODELING
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Regular Paper
Synthesizing SoS Concepts
for Use in Cost Modeling
Jo Ann Lane1, * and Ricardo Valerdi2
1
Center Systems and Software Engineering, University of Southern California, 941 West 37th Place, SAL Room 328, Los
Angeles, CA 90089-0781
2
Systems Engineering Advancement Research Initiative, Massachusetts Institute of Technology, 77 Massachusetts Avenue,
Building 41-205 Cambridge, MA 02139
SYNTHESIZING SoS CONCEPTS FOR USE IN COST MODELING
Received 1 February 2007; Accepted 5 June 2007, after one or more revisions
Published online in Wiley InterScience (www.interscience.wiley.com).
DOI 10.1002/sys.20078
ABSTRACT
Today’s need for more complex, capable systems in a short timeframe is leading many
organizations towards the integration of existing systems into network-centric, knowledge-
based system-of-systems (SoS). This presents new acquisition challenges in the area of cost
estimation because of the lack of commonly accepted definitions and roles. Software and
system cost models to date have focused on the software and system development activities
of a single system. When viewing the new SoS architectures, one finds that the cost associated
with the design and integration of these SoSs is not handled well, if at all, in current cost
models. This paper looks at commonly cited definitions of SoS, then evaluates these defini-
tions to determine if they adequately describe and converge on a set of SoS characteristics
in the areas of product, development process, and development personnel that can be used
to define boundaries and key parameters for an initial SoS cost model. Sixteen SoS definitions
are synthesized to provide reasonable coverage for different properties of SoSs. Two exam-
ples are used to illustrate key characteristics relevant to cost modeling. © 2007 Wiley
Periodicals, Inc. Syst Eng 10: 297–308, 2007
Key words: System of Systems definitions; cost estimation; cost modeling; FCS; GEOSS
*Author to whom all correspondence should be addressed (e-mail:
jolane@usc.edu; rvalerdi@mit.edu).
Systems Engineering, Vol. 10, No. 4, 2007
© 2007 Wiley Periodicals, Inc.
297298 LANE AND VALERDI
1. INTRODUCTION Cost modeling—the process of observation and vali-
dation of cost estimating relationships that cap-
A growing trend is emerging in industry and the U.S.
ture the relevant parameters that simulate the cost
Department of Defense (DoD) to “quickly” incorporate
of a system. Cost model development is often
new technologies and expand the capabilities of legacy
done using historical data collection and vali-
systems by integrating them with other legacy systems,
dated through statistical techniques.
Commercial-Off-the-Shelf (COTS) products, and new
Cost estimation—process of using a technique, pro-
systems. With this development approach, new activi-
cedure, or cost model to obtain an estimate for
ties are being performed to define the new architecture,
the cost associated with the development of a
identify sources to either supply or develop the required
system. This activity is often performed during
components, and eventually integrate and test these high
the bid and proposal stage of development in
level components. Along with this “system-of-systems”
response to a request for proposal.
(SoS) development approach, a new engineering disci-
pline is evolving to perform these activities: that of System
of Systems Engineering (SoSE), often performed by a lead 2. CURRENT SYSTEM AND SOFTWARE
system integrator, prime contractor, or government team COST MODELS
working in conjunction with support contractors. The Today, there are increasingly sophisticated models to
concepts of SoS and SoSE are often found embedded in support the estimation of the effort and schedule asso-
the quagmire of complex systems, enterprise systems, ciated with the development of the lower-level SoS
family of systems, DoD SoS concepts, systems engineer- component systems using three categories of parame-
ing SoS concepts, and computer science SoS concepts ters: product characteristics, process characteristics,
[Sheard, 2006]. However, there is an immediate need for and personnel characteristics [Boehm et al., 2005]. An
cost models to help estimate costs in this area due to the overview of these models is shown in Figure 1. For
growing SoS development trends. software development activities, there are the CO-
This paper begins by reviewing current system and COMO II [Abts, 2004], Cost Xpert [Cost Xpert Group,
software cost estimation models with respect to the SoS 2003], Costar [SOFTSTAR, 2006], PRICE-S [PRICE,
context. It then analyzes the most relevant SoS concepts 2006], SEER-SEM [Galorath, 2001], and SLIM
available and continues by defining a set of useful [QSM, 2006] cost models. At the single system level,
discriminators for determining the applicability of these there is COSYSMO [Valerdi, 2005] to estimate the
concepts to the area of cost modeling and estimation. system engineering effort and PRICE H [PRICE, 2006]
The goal of this analysis is not to develop yet another and SEER-H [Galorath, 2001] to estimate hardware
definition of system of systems, but rather to better development costs. For the implementation and integra-
understand what people are referring to as SoSs, how tion of COTS, there is COCOTS [Abts, 2004] and
these concepts might be converging, and how to better SEER-SEM [Galorath, 2001] to estimate the effort
support the planning and estimation of SoSE activities associated with the assessment, tailoring, and glue-code
in an initial SoS cost model. This approach advances implementation of COTS software products.
our understanding of the types of SoS characteristics However, none of these models includes SoSE-re-
that can be used to define a relevant set of parameters lated activities such as the definition of the SoS archi-
that estimate the costs associated with SoSE. To better tecture, the solicitation and procurement process for the
understand cost modeling and cost estimation in the SoS components, and the integration of the SoS com-
system development domain, the following definitions ponents into the SoS framework. Many organizations
are provided: often estimate these costs using a percentage of the
Figure 1. Suite of available cost models for software engineering, systems engineering, and hardware development.
Systems Engineering DOI 10.1002/sysSYNTHESIZING SoS CONCEPTS FOR USE IN COST MODELING 299
lower level system component development costs. that is planned up-front by a prime contractor or system
While this may be sufficient in some cases, it is not integrator. In others, an SoS is an ad hoc architecture
helpful in determining the individual drivers that affect that evolves over time, often driven by organization
SoSE cost and schedule. A better understanding of these needs, new technologies appearing on the horizon, and
drivers can help the acquirers and developers (a) reduce available budget and schedule. The ad hoc evolutionary
the risk of underestimating or overestimating the re- SoS architecture is more of a network architecture that
sources needed to support their investment in large grows with needs and available resources.
technology-intensive SoSs, (b) explore alternatives and In any case, users and nodes in the SoS network may
support trade-off activities, and (c) understand the sen- be either fixed or mobile. Communications may be
sitivities of the different cost drivers of SoSE. either point-to-point or broadcast. Networks may tie
This existing gap creates an opportunity to develop together other networks as well as nodes and users. SoS
cost models that address the SoSE activities for large component systems typically come and go over time.
SoS. But first it is essential to understand the relevant These component systems can operate both within the
aspects of SoS and the associated drivers of cost that SoS framework and independent of this framework. In
should be reflected in a cost model. a general sense, it is challenging to define clear bounda-
ries of an SoS because of its dynamic nature. Equally
3. WHAT IS AN SoS? challenging is the process of deciding what systems
appropriately deserve the SoS label because, depending
The earliest references in the literature to “systems on an individual’s system-of-interest, one person’s sub-
within systems” or “system of systems” can be found system may be another’s system, as stated in Rechtin
in Berry [1964] and Ackoff [1971]. These 1960–1970 [1991].
era SoS concepts are early insights into the evolution of
systems of today. Even though the term “system of 3.1. Current SoS-Related Concepts
systems” was not commonly used at the time, systems In preparation for the 2005 IEEE Systems, Man, and
of systems were being successfully developed and de- Cybernetics Conference, Jamshidi [2005] compiled a
ployed. These SoSs are represented by undersea sur- group of SoS definitions from a diverse group of
veillance and weapons systems such as the 1950s era authors. Each definition focused on distinct types, char-
Integrated Undersea Surveillance System (IUSS) and acteristics, and applications of SoSs. We provide a list
Sound Surveillance System (SOSUS), which signifi- that includes Jamshidi’s compilation as well as more
cantly expanded the capabilities of the World War II-era recent concepts appropriate to the context of SoS cost
Anti-Submarine Warfare (ASW) system [FAS, 2006; modeling. At first glance it is apparent that these defi-
IUSSCAA, 2006] described by Tom Clancy in the Hunt nitions have very little in common, possibly due to the
for Red October [Clancy, 1985], the Global Positioning diversity of interests and applications addressed. How-
System (GPS) [NAVSTAR, 2006], which is today con- ever, they help define the landscape of SoSs that is
sidered both as an SoS and a component system for relevant for cost modeling purposes. The definitions are
other SoSs, and military command and control centers. presented here in chronological order to show how the
As these types of integrated systems became more SoS-related concepts have evolved over time. Note that
common, system engineering professionals and re- each definition is identified by the author/organization
searchers began to define and study them as a special name for later reference.
class of systems. And, as the term has become a popular Eisner: Systems of systems are large geographically
way to represent a strategic and economic approach to distributed assemblages developed using centrally di-
enhancing existing system capabilities, there are now rected development efforts in which the component
an abundance of definitions. systems and their integration are deliberately, and cen-
A review of recent publications [Jamshidi, 2005; trally, planned for a particular purpose [Eisner, 1993].
USAF SAB, 2005] shows that the term “system-of-sys- Shenhar: An array system (system of systems) is a
tems” has a diverse range of meanings across different large widespread collection or network of systems func-
contexts. In the business domain, an SoS is the enter- tioning together to achieve a common purpose [Shen-
prise-wide integration and sharing of core business har, 1994].
information across functional and geographical areas. Manthorpe: In relation to joint warfighting, system
In the military domain, an SoS is a dynamic communi- of systems is concerned with interoperability and syn-
cations infrastructure and a configurable set of compo- ergism of Command, Control, Computers, Communi-
nent systems to support operations in a constantly cations, and Information (C4I) and Intelligence,
changing, sometimes adversarial, environment. In Surveillance, and Reconnaissance (ISR) Systems
some cases, an SoS may be a multisystem architecture [Manthorpe, 1996].
Systems Engineering DOI 10.1002/sys300 LANE AND VALERDI
Maier: Five principal characteristics are useful in propriate: operational independence, managerial inde-
distinguishing very large and complex but monolithic pendence, geographic distribution, emergent behavior,
systems from true systems-of-systems: operational in- and evolutionary development.... A system will be
dependence of the elements, managerial independence called a SoS when all or a majority of these charac-
of the elements, evolutionary development, emergent teristics are present [Sage and Cuppan, 2001].
behavior, and geographic distribution [Maier, 1996]. Gupta: The key success factor for good Lead System
This definition was later refined to reflect the fact that Integrators (LSIs) is not their internal capabilities to
several of the SoS characteristics above are often typical design, develop, and implement major defense systems,
characteristics of many systems: A system-of-systems but their ability to be visionaries and leaders who can
is an assemblage of components which individually coordinate, motivate, and work closely with a set of
may be regarded as systems and which possesses two co-contractors to achieve the ultimate objective in an
additional properties: operational independence of the optimal manner. The LSI must seek to perform its
components and managerial independence of the com- mission not by performing the bulk of the work in-
ponents [Maier, 1998]. house, but by seeking to leverage the work that is being
Kotov: Systems of systems are large-scale concur- done by others in a highly coordinated manner [Gupta,
rent and distributed systems that are comprised of com- 2003].
plex systems [Kotov, 1997]. Bergey, O’Brien, and Smith: An LSI is an agent with
Lukasik: SoS Engineering involves the integration the authority to acquire and integrate assets from a
of systems into systems of systems that ultimately variety of potential system suppliers on behalf of an
contribute to evolution of the social infrastructure organization that is acquiring a complex software-in-
[Lukasik, 1998]. tensive system. The LSI has the authority to contract
Krygiel: A system of systems is a set of different with and manage other suppliers on behalf of the ac-
systems so connected or related as to produce results quirer. A primary task of the LSI is to determine early
unachievable by the individual systems alone.… There in the integration cycle whether required software as-
is not one SoS but many SoSs. This arises from the need sets can be mined from existing assets, can be purchased
to use particular systems for different missions and the as COTS components, or need to be developed from
rate of change of circumstances and technology.… A scratch [Bergey, O’Brien, and Smith, 2003].
particular SoS may be configured and used for a period Keating et al: A metasystem, comprised of multiple
of days or weeks to support a mission-transient opera- embedded and interrelated autonomous complex sub-
tion. Other combinations of systems may be integrated systems that can be diverse in technology, context,
and sustained for longer periods of time.… Interoper- operation, geography, and conceptual frame…. These
ability is an essential requirement for a SoS [Krygiel, complex subsystems must function as an integrated
1999]. metasystem to produce desirable results in performance
Pei: System of Systems Integration is a method to to achieve a higher-level mission subject to constraints
pursue development, integration, interoperability, and [Keating et al., 2003].
optimization of systems to enhance performance in Defense Acquisition Guidebook (DAG): A set or
future battlefield scenarios [Pei, 2000]. arrangement of interdependent systems that are related
Carlock and Fenton: Information-intensive SoSs or connected to provide a given capability. The loss of
(are) typically composed of an evolving mix of legacy any part of the system will degrade the performance or
and new systems.… Developing an SoS, especially one capabilities of the whole.… The consideration of sys-
involving a number of legacy systems is a far more tem of systems engineering should include the follow-
complex job than developing a stand-alone system … ing factors or attributes: larger scope and greater
with priorities focused on seamless interoperability and complexity of integration efforts; collaborative and dy-
acceptable performance. Enterprise systems of systems namic engineering; engineering under the condition of
engineering is focused on coupling traditional systems uncertainty; emphasis on design optimization; continu-
engineering activities with enterprise activities of stra- ing architectural reconfiguration; simultaneous model-
tegic planning and investment analysis [Carlock and ing and simulation of emergent system of systems
Fenton, 2001]. behavior; and rigorous interface design and manage-
Sage and Cuppan: Systems formed from a variety ment [DoD DAG, 2004].
of component systems: newly engineered from the United States Air Force (USAF) Scientific Advisory
“ground-up” custom systems, potentially tailored exist- Board: A configuration of systems in which component
ing COTS systems, and existing or legacy systems.… systems can be added/removed during use; each pro-
These systems have five characteristics [Maier, 1996] vides useful services in its own right; and each is
that make the system of systems designation most ap- managed for those services. Yet, together they exhibit a
Systems Engineering DOI 10.1002/sysSYNTHESIZING SoS CONCEPTS FOR USE IN COST MODELING 301
synergistic, transcendent capability. Related to this, 3.2. Classification of SoS Definitions
SoSE is defined as: The process of planning, analyzing,
On closer inspection of these definitions, what is most
organizing, and integrating the capabilities of a mix of
apparent is that they each have a different application
existing and new systems into a system-of-systems
domain such as military, private enterprise, or educa-
capability that is greater than the sum of the capabilities
tion. They also focus on different characteristics of the
of the constituent parts. This process emphasizes the
SoS. Table I shows the characteristics identified in each
process of discovering, developing, and implementing
of the 16 definitions. The most popular characteristics
standards that promote interoperability among systems
used across the 16 definitions are “emergent behav-
developed via different sponsorship, management, and
ior/synergistic/higher-level purpose” (11), “complex”
primary acquisition processes [USAF SAB, 2005].
(6), “interoperable systems” (6), and “mix of existing,
Northrop et al: A system comprising independent,
new, or diverse systems” (6).
self-contained systems that, taken as a whole, satisfy a
The SoS characteristics have also been classified
specified need [Northrop et al., 2006].
using the three categories of cost model parameters:
It should be noted that the above 16 SoS-related
operational/product, implementation/process, and per-
definitions include SoS, SoS integration, SoSE, and
sonnel. The operational/product category includes SoS
LSI. In order to better understand these concepts, we
architectural characteristics, SoS composition, and
tease out specific relevant attributes for further analysis
characteristics of the SoS component systems from
in the next section.
either a development or operational perspective. The
Table I. Characteristics of Definitions
Systems Engineering DOI 10.1002/sys302 LANE AND VALERDI
Table II. Mapping of SoS Charactericstics to Cost Model Parameter Categories
implementation/process category includes the plan- that were analyzed focus on either the product or proc-
ning, management, and engineering processes used to ess characteristics of SoS. In fact, most definitions
develop the SoS. The personnel category describes the address both. However, only two of the 16 definitions
personnel skills and focus needed to develop the SoS. address the personnel characteristics of the SoS, a cru-
Table II shows the relationships between the SoS char- cial function typically played by the prime contractor,
acteristics identified in the selected definitions and the system integrator, or government team.
cost model parameter categories. Rather than merging these definitions and attempt-
Using Tables I and II, one can look at the mapping ing to develop a “one-size-fits-all” definition for SoS, a
of the selected definitions to the cost model parameters more appropriate approach is to analyze them to deter-
categories. As shown in Table III, most of the definitions mine a common set of SoS characteristics that can be
synthesized with our SoS cost model goals and later
used to (a) define an appropriate set of cost model
drivers and (b) identify candidate SoSs to support
Table III. Focus of Selected SoS-Related Definitions model calibration and validation. A set of discrimina-
tors has been developed to help distinguish features and
clarify the distinction between definitions that best in-
form cost modeling. These discriminators are then ap-
plied to two distinct government SoS programs.
4. MODEL DEVELOPMENT PROCESS
One of the best-known quality gurus, W. Edwards Dem-
ing, popularized the teaching “If you can’t measure it,
you can’t manage it.” In order for this teaching to be
effective, Deming must have assumed that there was a
good definition for whatever was being measured. In
the case of cost estimation, there needs to be: (1) clear
definitions of what is being estimated, (2) clearly de-
fined cost model parameters, and (3) established count-
ing rules for size factors. Consider the following
example: when attempting to estimate the cost of SoSE
effort, it is essential to (1) clarify what is meant by SoS,
(2) define parameters relevant to SoS and the associated
Systems Engineering DOI 10.1002/sysSYNTHESIZING SoS CONCEPTS FOR USE IN COST MODELING 303
engineering activities, and (3) develop counting rules
for the relevant size factors that influence cost in this
setting. Defining what is being estimated has led us to
an equivalent corollary to Deming’s teaching: “If you
can’t describe it, you can’t estimate it.” This statement
is an antecedent to Deming’s teaching and creates the
following logical sequence: describe → estimate →
measure → manage. The current focus is to describe
SoSs in order to facilitate the estimation and eventually
management of SoS development activities.
In an effort to determine criteria relevant to the
development of a SoS cost model, it is important to
understand the process of developing a cost estimation
model. The University of Southern California (USC)
Figure 3. Typical cost model life cycle.
Center for Systems and Software Engineering (CSSE)
has developed and continues to use an eight-step proc-
ess for creating cost models, as illustrated in Figure 2
5. DISCRIMINATORS FOR EVALUATING
[Boehm et al., 2000]. SoS DEFINITIONS
Step 1 in this methodology, involves the identifica-
tion of a market for a cost estimation model. This is During the process of analyzing the different SoS defi-
typically a group of stakeholders that will support the nitions, a set of discriminators were developed to help
model development effort with funding, expertise, and filter existing definitions. The SoS definitions were
data. Without these three forms of support it is very viewed in light of the three cost model parameter cate-
difficult to develop an industry validated cost model in gories: product, process, and personnel. Key aspects of
a research setting. For a detailed explanation of the the definitions that fell in these areas were the economi-
subsequent steps in Figure 2, see Boehm et al. [2000]. cally oriented SoS stakeholders, system integrators re-
The full life cycle of a cost estimation model might sponsible for the development of the SoS and the
be viewed as a cyclical process as shown in Figure 3. development processes they used to develop the SoS,
Once a market for the model has been identified and the SoS architecture characteristics, and component system
characteristics. The following describes the SoS cost
stakeholder needs are understood, a spiral process be-
modeling discriminators extracted from the SoS defini-
gins that involves the development, use, and refinement
tions:
of model artifacts. The role of industry practitioners is
crucial in this process since they provide the real-life Discriminator #1—Economically-Oriented SoS
examples of SoSs and data to validate cost estimating Stakeholders: For an SoS cost estimation model
relationships. to be of value, it is important that there exist
strategically-oriented organizations that have a
need for this type of information. These organi-
zations include clients that will pay for the system
and signoff on major milestones as well as user
communities that will be responsible for SoS
operation and sustainment. This discriminator is
important for the development of the SoS cost
model since the success of this model depends
upon historical data from motivated stakeholders
to support model calibration and validation.
Discriminator #2—SoSE Responsibilities and Proc-
esses: The program must identify a single (or an
obvious set of) lead systems integrator(s), prime
contractor, or government team that is responsi-
ble for the definition of the SoS architecture and
the total component system integration and test
Figure 2. USC CSSE cost model development methodology. activities at the SoS level, often referred to as
Systems Engineering DOI 10.1002/sys304 LANE AND VALERDI
SoSE. It is important that this organization have 6. SYNTHESIZING DEFINITIONS
complete technical oversight over the entire SoS
and SoS component suppliers, as well as the Now that the different definitions of SoS have been
engineering processes at the SoS level. This en- described and the discriminators for evaluating them
sures that there is an organization responsible for have been presented, the next step is to synthesize these
defining an overall architectural vision for the definitions and identify what each of them contributes
SoS while maintaining the architectural concepts in the domain of cost modeling. The more discrimina-
throughout the development, integration, and test tors addressed in a given definition, the more useful the
definition is for purposes of cost modeling. The results
phases. This is in contrast to SoSs that are more
of the evaluation of these definitions are presented in
evolutionary and collaborative in nature and
Table IV. Note that some of the discriminators were
based on much shorter-term strategies that can
very strongly addressed in a given definition and these
change frequently due to business needs and
are indicated with “X” in Table IV. Other discriminators
performance (for example, strategic alliances and were sometimes implied in a given definition and these
cost-sharing partnerships as described in Fried- are indicated with “x”.
man [2005]). All but two of the SoS-related definitions in Table
Discriminator #3—Component-Based SoS Archi- IV meet at least two of the cost model discriminators,
tecture: The SoS is primarily the integration of a and, together, the 16 definitions provide reasonable
set of component systems that work together to coverage of the product, process, and personnel charac-
provide emergent behaviors not provided by any teristics of interest in cost modeling. To test these dis-
single component system within the architecture. criminators, two systems generally thought to be
The key architecture feature in an SoS is the system-of-systems are analyzed in terms of consistency
framework that supports the integration of exist- and relevancy with respect to the selected criteria: the
ing as well as new component systems. The char- US Army’s Future Combat System and the Group on
acteristics of this architecture determine the Earth Observation’s Global Earth Observation System
ease/difficulty in integrating SoS component sys- of Systems.
tems in both initial development and dynamic Future Combat System (FCS): As stated in an Octo-
operational scenarios. ber 2004 white paper [US Army, 2004], “FCS is a joint
Discriminator #4—Component System Inde- networked system of systems…connected via an ad-
pendence: Each of the component systems envi- vanced network architecture that will enable levels of
sioned within the SoS must be independent with joint connectivity, situational awareness and under-
respect to development, maintenance, and opera- standing, and synchronized operations heretofore un-
tion. This provides for clear boundaries for the achievable. FCS will operate as a SoS that will network
various cost models that can be used to estimate existing systems, systems already under development,
total SoS development costs, This view, focusing and systems to be developed to meet the requirements
on one of the key aspects identified in several SoS of the Army’s Future Force Unit of Action.”
definitions, minimizes the potential overlap be- The FCS is currently being planned and managed by
an LSI team comprised primarily of Boeing and Science
tween the SoS cost estimation model for SoSE
Applications International Corporation (SAIC), work-
activities and the system engineering and soft-
ing in partnership with the US Army. The FCS capabili-
ware cost estimation models used to estimate the
ties are being provided through an incremental, fixed
implementation of SoS-enabling functionality in
cost and schedule process [Dalrymple, 2006]. A sum-
the component systems. mary of the four discriminators for the FCS is provided
in Table V. All four discriminators are easily identifiable
From the cost modeling perspective, an acceptable for an SoS cost model to estimate LSI effort, making
SoS definition must contain at least one of the discrimi- FCS well suited for SoS cost modeling tools.
nators. No single definition must meet all four discrimi- Global Earth Observation System of Systems
nators, but portions of each definition can be borrowed (GEOSS): As stated on the Environmental Protection
to help guide cost model developers through the next Agency’s (EPA) GEOSS website [EPA, 2007], the
step of creating a meaningful model. By using these key “earth observation systems consist of measurements of
prevalent discriminators, they can more easily identify air, water, and land made on the ground, from the air,
and converge on a minimal, yet necessary set of cost or from space. Historically observed in isolation, the
model parameters and the range of values needed to current effort is to look at these elements together and
address the various types and views of SoSs. to study their interactions.” Over 100 datasets, models,
Systems Engineering DOI 10.1002/sysSYNTHESIZING SoS CONCEPTS FOR USE IN COST MODELING 305
Table IV. Results of Definition Synthesis
decision support tools, and programs are identified as sharing of data and information from a variety of sys-
part of GEOSS [EPA, 2007]. tems, with no significant plans to tightly integrate the
Contrary to FCS, the four discriminators are not yet systems or data. While there is distinct component
clearly defined or easily determined, possibly as a result independence, there is no primary set of stakeholders
of the distributed, decentralized, loosely-coupled, and or LSI organizations. Rather, it is designed to be an ad
emergent nature of GEOSS. hoc evolutionary SoS that is supported by member
For instance, the complicated structure of the Group organizations. The member organizations can be
on Earth Observations (GEO) involves over 115 coun- viewed as both the integrators and end users. While
tries and participating organizations [GEO, 2007]. The GEOSS is a valid SoS by many of the SoS definitions
structure of GEO does not allow for a single organiza- identified earlier (e.g., Lukasik, Maier, Sage, and Cup-
tion to act as a majority stakeholder. The primary focus pan), the discriminators indicate that GEOSS is not a
of the GEO appears to be worldwide collaboration and good candidate for cost modeling because of its decen-
tralized nature, technical architecture, and loose
boundaries. A summary of the four discriminators for
GEOSS is provided in Table VI.
Table V. FCS Discriminators The FCS and GEOSS provide two examples of SoS
developments that behave differently when compared
in terms of our four discriminators. Returning to the
earlier corollary “if you can’t describe it, you can’t
estimate it,” one sees with the four discriminators that
the FCS SoSE effort is a good candidate to support
SoSE cost model development and benefit from the use
of the cost model to estimate costs going forward. On
the other hand, GEOSS SoS design and integration is
too loosely defined at this point in time to support cost
model development or to be able to benefit from the use
Systems Engineering DOI 10.1002/sys306 LANE AND VALERDI
Table VI. GEOSS Discriminators
of a cost model to estimate costs. The GEO organization component systems that are operationally and man-
may find an SoS cost model beneficial in the future if agerially independent of each other, and the evolution-
the SoS evolves to include tighter integrations of sub- ary nature of the SoS. Finally, the four discriminators
systems, tools, stakeholders, and architectures. will also help in identifying more SoSs, other than FCS,
that can be used to test these drivers and calibrate a cost
7. CONCLUSIONS AND NEXT STEPS model for SoSE.
A crucial first step in developing relevant models in-
REFERENCES
volves the identification of what is being modeled. This
paper synthesized existing SoS definitions and devel- C. Abts, Extending the COCOMO II software cost model to
oped concepts to evaluate different properties of SoS. estimate effort and schedule for software systems using
It also showed that the product, process, and personnel Commercial-Off-The-Shelf (COTS) software compo-
characteristics needed for cost model development are nents: The COCOTS model, Ph.D. Dissertation, Univer-
adequately covered by 16 existing definitions. More- sity of Southern California, 2004.
over, these characteristics have enabled the develop- R. Ackoff, Towards a system of systems concept, Manage-
ment of four discriminators for evaluating the relevance ment Sci 17(11) [Theory Series] (July 1971), 661–671.
of SoS definitions in the area of cost modeling. These J. Bergey, L. O’Brien, and D. Smith, Application of options
discriminators were useful for evaluating two distinct analysis for reengineering in a lead system integrator
SoSs: FCS and GEOSS. The FCS discriminators are environment, CMU/SEI-2003-TN-009, Software Engi-
well defined and well suited for cost estimation, making neering Institute, Pittsburgh, PA, 2003.
it a good candidate for cost model development. B. Berry, Cities as systems within systems of cities, The
GEOSS, however, is currently not a good candidate for Regional Science Association Papers, Volume 13, 1964.
cost model development because of its loosely-defined http://www.regionalscience.org/
stakeholders, disparate SoSE, and lack of an overall B. Boehm, C. Abts, A. Brown, S. Chulani, B. Clark, E.
architecture. Horowitz, R. Madachy, D. Reifer, and B. Steece, Software
cost estimation with COCOMO II, Prentice Hall, Engle-
The synergy of the 16 definitions, their coverage of
wood Cliffs, NJ, 2000.
three relevant cost model parameter categories, and the
B. Boehm, R. Valerdi, J. Lane, and A. Brown, COCOMO suite
development of the four discriminators will aid SoS
methodology and evolution, CrossTalk 18(4) (April
cost model developers in the next steps as they proceed
2005), 20–25.
to develop and refine relevant cost and size drivers in P. Carlock and R. Fenton, System of Systems (SoS) enterprise
the SoS context. The desired cost and size drivers are systems for information-intensive organizations, Syst Eng
based on the SoS architecture characteristics (product), 4(4) (2001), 242–261.
processes used for design and integration/test (process), T. Clancy, Hunt for Red October, Collins, New York, 1985.
and SoSE experience and capabilities (personnel). The Cost Xpert Group, Inc., Cost Xpert 3.3 Users Manual, San
more prevalent SoS characteristics identified in the Diego, CA, 2003.
analysis of the 16 definitions indicate that an SoSE cost E. Dalrymple, Future combat systems SoS characteristics and
model should capture aspects related to SoS emergent critical success factors, Proc USC CSSE Convocation,
behaviors, higher levels of architecture and process October 2006. http://sunset.usc.edu/events/2006/
complexity, the interoperability of a mix of evolving CSSE_Convocation/pages/program.html
Systems Engineering DOI 10.1002/sysSYNTHESIZING SoS CONCEPTS FOR USE IN COST MODELING 307
Department of Defense (DoD), Defense Acquisition Guide W. Manthorpe, The emerging joint system of systems: A
(DAG), version 1.6, http://akss.dau.mil/dag/ accessed on systems engineering challenge and opportunity for APL,
December 27, 2006. John Hopkins APL Tech Dig 17(3) (1996), 305–310.
H. Eisner, RCASSE: Rapid Computer-Aided Systems of NAVSTAR Global Positioning System Joint Program Office,
Systems Engineering, Proc 3rd Int Symp Natl Council http://gps.losangeles.af.mil/, accessed December 6, 2006.
Syst Eng, NCOSE, 1993, Vol. 1, pp. 267–273. L. Northrop, P. Feiler, R. P. Gabriel, J. Goodenough, R.
Environmental Protection Agency (EPA), Global Earth Ob- Linger, T. Longstaff, R. Kazman, M. Klein, D. Schmidt,
servation system of systems tools, http://www.epa.gov/ K. Sullivan, and K. Wallnau, Ultra-large-scale systems:
geoss/index.html, 2007. The software challenge of the future. Pittsburgh, PA:
Federation of American Scientists (FAS), Integrated Under- Software Engineering Institute, Carnegie Mellon Univer-
sea Surveillance System (IUSS), http://www.fas.org/ sity, 2006.
irp/program/collect/iuss.htm, accessed December 27, R. Pei, Systems of Systems Integration (SoSI)—a smart way
2006. of acquiring Army C4I2WS systems, Proc Summer Com-
T. Friedman, The world is flat: a brief history of the twenty- put Simul Conf, 2000, pp. 574–579.
first century, Farrar, Straus and Giroux, New York, 2005. PRICE, Program affordability management,
Galorath, Inc., SEER-SEM users manual, El Segundo, CA, http://www.pricesystems.com, accessed December 30,
2001. 2006.
Group on Earth Observations (GEO), http://www.earthobser- QSM, SLIM-estimate, http://www.qsm.com/slim_esti-
vations.org, accessed May 18, 2007. mate.html, accessed December 30, 2006.
A. Gupta, Role and importance of lead system integrator in
E. Rechtin, System architecting: Creating and building com-
context of new air operations centers, MIT Sloan School
plex systems, Prentice-Hall PTR, Englewood Cliffs, NJ,
of Management, Cambridge, MA, 2003.
1991.
IUSS-Caesar Alumni Association (IUSSCAA), IUSS His-
A. Sage and C. Cuppan, On the systems engineering and
tory, http://www.iusscaa.org/history.htm, accessed De-
management of systems of systems and federations of
cember 27, 2006.
systems, Inform Knowledge Syst Management 2 (2001),
M. Jamshidi, System-of-systems engineering—a definition,
325–345.
I EEE 2005 Int Conf Syst Man Cybernet,
http://ieeesmc2005.unm.edu/SoSE_Defn.htm, accessed S. Sheard, Systems of systems necessitates bridging systems
May 2005. engineering and complex systems sciences, presentation
C. Keating, R. Rogers, R. Unal, D. Dryer, A. Sousa-Poza, R. given at Second Annual Systems of Systems Engineering
Safford, W. Peterson, and G. Rabadi, System of systems Conference, Defense Acquisition University, Ft. Belvoir,
engineering, Eng Management J 15(3) (September 2003), VA, July 25–26, 2006.
36–45. A. Shenhar, A new systems engineering taxonomy, Proc 4th
V. Kotov, Systems of systems as communicating structures, Int Symp Natl Council Syst Eng, NCOSE, 1994, Vol. 2,
Hewlett Packard Computer Systems Laboratory Paper pp. 261–276.
HPL-97-124, 1997, pp. 1–15. SOFTSTAR, http://softstarsystems.com, accessed December
A. Krygiel, Behind the wizard’s curtain: An integration 30, 2006.
environment for a system of systems, C4ISR Coopera- United States Air Force Scientific Advisory Board (USAF
tive Research Program, http://www.dodccrp.org/html4/ SAB), Report on system-of-systems engineering for Air
bookdownloads.html, 1999. Force capability development; Public Release SAB-TR-
S. Lukasik, Systems, systems of systems, and the education 05-04, 2005. https://acc.dau.mil/Get Attachment.
of engineers, Artif Intell Eng Des Anal Manuf 12(1) aspx?id=150224&pname=file&lang=en-US&aid=28788
(1998), 55–60. US Army, FCS 18+1+1 White Paper, http://www.army.mil/
M. Maier, Architecting principles for systems-of-systems, fcs/index.html, 2004.
Proc INCOSE Symp, 1996. R. Valerdi, The Constructive Systems Engineering Cost
M. Maier, Architecting principles for systems-of-systems, Model (COSYSMO), Ph.D. Dissertation, University of
Syst Eng 1(4) (1998), 267–284. Southern California, Los Angeles, May 2005.
Systems Engineering DOI 10.1002/sys308 LANE AND VALERDI
Jo Ann Lane is currently a Principal at the University of Southern California Center for Systems and
Software Engineering conducting research in the area of system of systems engineering. In this capacity,
she is currently working on a cost model to estimate the effort associated with system-of-system
architecture definition and integration. She is also a part time instructor teaching software engineering
courses at San Diego State University. Prior to this, she was a key technical member of Science
Applications International Corporation’s Software and Systems Integration Group responsible for the
development and integration of software-intensive systems and systems of systems. Ms. Lane received
her B.A. in Mathematics and her M.S. in Computer Science from San Diego State University and is
currently a Ph.D. candidate in Systems Architecting and Engineering at the University of Southern
California. Ms. Lane is currently a member of INCOSE and IEEE.
Ricardo Valerdi is a Research Associate at the Lean Aerospace Initiative and the Systems Engineering
Advancement Research Initiative at MIT and a Visiting Associate at the Center for Systems and Software
Engineering at USC. Formerly he was a Member of the Technical Staff at the Aerospace Corporation in
the Economic & Market Analysis Center and a Systems Engineer at Motorola and at General Instrument
Corporation. He earned his B.S. in Electrical Engineering from the University of San Diego, and his M.S.
and Ph.D. from USC. He is a member of INCOSE and serves as Associate Director for International
Growth.
Systems Engineering DOI 10.1002/sysYou can also read
