Technology Overview for Ethernet Switching Fabric
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
G00249268
Technology Overview for Ethernet Switching
Fabric
Published: 16 May 2013
Analyst(s): Caio Misticone, Evan Zeng
The term "fabric" has been used in the networking industry for a few years,
but confusion remains among networking professionals. This research goes
beyond the "marketecture" to define data center Ethernet fabric, including
the main use cases and alternative technologies.
Key Findings
■ Ethernet switching fabric is the next logical evolution from the traditional Ethernet network for
very large service providers, carriers, large enterprises and highly virtualized data centers where
virtual machine (VM) mobility, on demand network services and cloud computing are high
priorities.
■ Gartner has seen customers adopting virtual chassis switch/clustering technologies to build
fabric-like networks at a lower cost than full fabric implementations. Next, they may consider
skipping current fabric approaches and move directly to software-defined networking (SDN).
■ Server virtualization, VM mobility and cloud computing are driving the move toward a fabric-
based network solution.
■ Implementation of current data center switching fabric solutions will likely lead to vendor lock-
in, as most existing implementations are based on proprietary versions of standard protocols,
such as TRILL and SPB.
Recommendations
■ Confirm that your vendor offers a migration path to a fabric-based architecture, if required.
■ Evaluate short-term alternatives (such as virtual chassis switch/clustering solutions), if moving
from a traditional LAN to a network fabric-based offering is not a required option. SDN and
hybrid SDN implementations using overlay technologies, such as Virtual eXtensible LAN
(VXLAN) and Network Virtualization using Generic Routing Encapsulation (NVGRE), should be
considered as alternative future solutions to improve business agility.
■ Expect to deploy a single-vendor data center switching fabric solution.What You Need to Know
Evolution of the Data Center Infrastructure
The data center has undergone several major architectural changes in the past few decades,
evolving from single monolithic compute and storage resource and distributed client/server
architecture to cloud-based computing. In this decade, it is undergoing another big architectural
transition, from a simple server virtualization environment to a highly virtualized data center. "Eight
Key Impacts on Your Data Center LAN Network" outlines all the key trends in the data center LAN
network; here, we look at the four primary drivers of the evolution to Ethernet switching fabric:
■ Resource pooling and fabric-based infrastructure: As compute evolves into virtual resource
pools, it is time for network and storage to follow. Only after that will end users be able to
choose pieces from different resource pools to form their choice of self-built and on-demand
infrastructure. See "Clearing the Confusion about Fabric-Based Infrastructure: A Taxonomy" for
our definition of fabric-based infrastructure.
■ Cloud computing: Compute on-demand isn't very useful unless the associated storage and
network are also on-demand. This makes end users embrace compute, storage and network
resources, and build on-demand capabilities.
■ East-west traffic: East-west traffic is dominant in the data center network, due to a number of
contributing factors: application deployment model changes- breaking down monolithic client/
server applications into tiered and modular Web-based service-oriented architecture (SOA)
composite applications, big data, and network/SAN convergence.
■ VM mobility: VMs are evolving from a static resource residing only on a fixed physical host to a
dynamic resource that moves from server to server in near real time. This makes all associated
resources, such as storage and network, follow the mobility requirements and be VM-policy-
aware to automate associated policies on the fly with VMs.
Limitations of Current Data Center Network
The main limitations of the current data center Network are:
■ Traditional three-tier network architecture is optimized for north-south traffic, but not for the
increasingly predominant east-west traffic in enterprise data centers. With the creation of a
virtual switch layer for VM access, physical aggregation layer switches become less necessary
and largely counterproductive.
■ Spanning Tree Protocol (STP) is widely implemented at data centers to prevent potential
network loops. It can block 50% or more of switch ports in a redundant data path; therefore, it
makes very inefficient use of network infrastructure.
Slow and frequent convergence results in degraded performance. Once any link gets removed/
added to the network or any network topology changes, STP halts all traffic through the network
and must recalculate the single path among all switches in the network before flowing traffic again.
This process generally takes tens of seconds to minutes on all switch links, based on various
Page 2 of 10 Gartner, Inc. | G00249268network sizes. It is unacceptable for mission-critical workloads to lose connectivity in the data path
for a few seconds. Its successors' Rapid Spanning Tree Protocol (RSTP) or Multiple Spanning Tree
Protocol (MSTP) can be used to reconverge and recover in a more timely manner, but will still
compromise data center efficiency and performance.
■ Each switch has its own control and management planes, without shared common
configuration and policy information between switches. This is very complex in terms of
operation and maintenance.
■ Once a VM migrates across physical servers, the VM policy on a specific switch port does not
migrate on the fly, together with its VM, to the destination switch port. It always requires manual
configuration. This approach is not agile and tolerant in today's network.
Analysis
The increasing adoption of virtualization, cloud computing and server/storage convergence is
requiring a rearchitecture of the data center environment. It is also introducing new technological,
operational and organizational challenges, while adding new requirements to the existing data
center network, such as scalability, simplification and flexibility.
To support such a data center vision, the whole infrastructure needs to evolve, especially the
network. Because Ethernet is the most widely accepted network standard in the data center, it
needs to evolve from interconnecting physical switches to one Ethernet fabric that is VM-aware and
behaves just like one integrated switch to support data center transformation.
Ethernet fabric can be one- or two-tier Ethernet architectures in the data center (see Figure 1). The
differences between those two forms are one- or two-hop forwarding across the fabric, and how
edge ports connect together into one switch. In general, Ethernet fabric is composed of hardware
and software components to make multiple interconnecting switches behave like a single switch.
Gartner, Inc. | G00249268 Page 3 of 10Figure 1. Ethernet Fabric Architecture
Ethernet Fabric
Management
Tier Core Switches
Edge
Switches
Server Server Storage
VM VM Virtual Storage
VM VM
VM VM Virtual Storage
Source: Gartner (May 2013)
Evolving the data center network into a network fabric requires changes in current network design
practices — moving from the conventional three-tier architecture to a one- or two-tier architecture,
as well as making use of new technologies such as TRILL and SPB, and possibly new types of
switches with larger Media Access Control (MAC) address tables. The technology and product
evolution will be significant, and will not, in all cases, be backward-compatible, which will require a
planned road map to ensure a smooth evolution or an eventual "rip and replace" of the current
infrastructure. There are vendors that may allow some components of their hierarchical design to be
reused for an Ethernet fabric architecture (for example, with a software upgrade). Enterprises should
consider the migration paths for any upgrade in the data center, because there are several major
vendor solutions that may require a rip and replace, even if the vendor is the incumbent. Ethernet
fabric should also offer standard base interfaces and protocols to interoperate with networks
running on legacy protocols such as STP.
Technology Description
Historically, the term "switching fabric" was used to describe the backplane and switching capacity
of a single LAN switch. In the data center, fabric meant Fibre Channel or InfiniBand. Today, the term
is generally used by vendors to describe the whole set of interconnected switches and their
Page 4 of 10 Gartner, Inc. | G00249268interconnections with the implementation of Layer 2 multipathing technologies based on TRILL,
SPB and other proprietary versions.
As shown in Figure 1, Ethernet fabric is a logical entity composed of a number of hardware or virtual
switches, various software components, and a single management platform. It is not only designed
to interconnect physical servers and storage, but also VMs and virtual storage resources. Ethernet
fabric should meet the following requirements:
■ Control plane: The fabric can have either a centralized or distributed control plane in every
switch node. An intelligent forwarding protocol is used that addresses the limitations of STP,
including inefficient use of bandwidth and slow convergence. Also, it can support an integrated
Layer 2/Layer 3 forwarding across all nodes in the fabric.
■ Data plane: The fabric data path supports equal cost multipath forwarding at Layer 2 and data
always takes the shortest path using multiple Inter-Switch Link (ISL) connections without loops.
Also, all the server-facing ports in edge switches focus on user profile discovery and policy
automation, and there should be a single shared-state table across the fabric for these ports.
For the network-facing ports, the focus is on bandwidth aggregation, and the ports are
transparent to user profile and policy.
■ Redundancy: The fabric supports fast convergence if any network topology changes or links
go up or down.
■ Automation: The fabric should require a very simple setup or zero-touch deployment, and is
able to automate network configuration and policy on a large scale.
■ Management: The fabric supports a single point of management.
■ Flat architecture: The fabric supports one- or two-tier architecture with deterministic latency
from any port to any port within the fabric.
Ethernet fabric may or may not meet the following requirements, depending on your business and
technical requirements:
■ Low latency: High-frequency trading (HFT), cluster/computing modeling or deep analytics
applications require the lowest possible latency.
■ Lossless: Convergence of Fibre Channel and Ethernet over a single network requires a
guaranteed delivery of FCoE frames using data center bridging (DCB) to ensure a lossless
operation.
■ Interoperability: The fabric should provide a standard base interface to interoperate with third-
party devices and networking equipment.
■ VM awareness: The fabric is a VM-aware network and supports seamless VM mobility across
physical servers with automated network policy and services on the fly, together with the
moving VMs.
Gartner, Inc. | G00249268 Page 5 of 10Technology Definition
Ethernet fabric is an Ethernet construct in which multiple physical switches interconnect with each
other and combine with hardware and software components to form a more elastic, automated and
simpler switch network that behaves like a single switch. Ethernet fabric is not a switch cluster
technology, and runs integrated Layer 2/Layer 3 standard base or proprietary protocols across all
its switching fabric member nodes. Besides the classic Ethernet functions, Ethernet fabric has the
following new characteristics:
■ Support equal-cost multipath forwarding at Layer 2/Layer 3 with fine-grained, flow-based load
balancing.
■ Replace STP with Layer 2 Multipathing protocols that can be either standard base or
proprietary. These protocols do not have the same limitations as STP and can support Layer 2
Equal Cost Multipath (ECMP) and faster convergence once the topology changes or any switch
link is up or down.
■ The whole Ethernet fabric collapses the legacy three-tier network into a one- or two-tier
architecture across the full data center and with deterministic latency from any port to any port.
■ All the server-facing switch ports can be either fully VM-aware or transparent with a single
shared, scalable-state table for all ports.
■ All devices and ports in the fabric can be accessed and managed by a single point.
Ethernet fabric can be integrated with a VM provisioning and management system to provide
multitenancy and policy automation for VMs, which can lead to a worse vendor lock-in scenario.
Standards
There are some standards and vendor-specific technologies that are usually involved and should be
considered in a data center Ethernet switching fabric ecosystem. Some of them include:
■ Layer 2 Multipathing: TRILL, SPB, Cisco FabricPath, Juniper Networks QFabric System,
Brocade VCS
■ Multichassis Link Aggregation (MC-LAG): Cisco Virtual Switching System (VSS)/vPC, HP
Intelligent Resilient Framework (IRF), Juniper Virtual Chassis
■ Storage/Network Convergence Protocols: FCoE
■ Lossless Ethernet for Converged Infrastructure: DCB
Uses
In reviewing Gartner client inquiries, we have identified three broad use cases for Ethernet switching
fabrics:
■ The service provider's data center, where network automation, scalability, performance and
manageability are critical
Page 6 of 10 Gartner, Inc. | G00249268■ The enterprise's data center, with a highly virtualized and cloud-oriented environment that
requires deterministic end-to-end latency, network automation and on-demand services
■ Any sizable data center that wants to enhance network automation, requires on-demand
network services, and gets rid of STP and facilitates network provisioning
Benefits and Risks
Benefits:
■ Fabrics can simplify network operations by moving from a physical three-tier architecture to one
or two tiers, providing a single point of management with high automation and treating the
network as one logical switch.
■ Fabrics provide deterministic latency for east-west traffic: In a traditional three-tier network,
latency increases as traffic flows through each individual hop. This latency may not impact the
movement of traditional best-effort applications, but it certainly impacts latency-sensitive
applications (such as workload mobility and enterprise storage).
■ Fabric provides multipath technology that increases the bandwidth by allowing all paths to a
certain destination to be active at the same time, improving overall performance and reliability
while eliminating downtime due to slow reconvergence times.
■ Automation port profile migration for VM mobility makes Ethernet fabric a very convenient
solution to operate in highly virtualized and cloud-oriented data centers, where agility is crucial.
Risks:
■ Because every fabric vendor solution has some sort of proprietary feature, vendor lock-in is a
real possibility in the data center switching fabric market. Selecting a single vendor's proprietary
fabric solution is similar to the way most organizations standardize on a single core data center
network vendor today.
■ Implementing a switching fabric can make your data center more complex, if your vendor
doesn't offer you a clear migration path from a traditional data center network. This is because
the Ethernet fabric runs over specific proprietary protocols and has a clear boundary with
traditional data center networks.
Technology Alternatives
Gartner sees two main technology alternatives to fabric-based solutions:
■ Clustering Solutions: This technology has been used by many enterprises that want to achieve
resiliency, redundancy and management simplification without necessarily moving toward the
deployment of a fabric solution. Most vendors have their own proprietary versions of clustering
or MC-LAG solutions (for example, Cisco VSS/vPC, HP IRF and Juniper Virtual Chassis).
Gartner, Inc. | G00249268 Page 7 of 10■ SDN: Gartner expects emerging technologies such as SDN to become real and mature
alternatives to data-center-switch-fabric-based solutions in two to three years. As defined in
"Ending the Confusion About Software-Defined Networking: A Taxonomy," SDN is considered a
new way to design, build and operate networks. It decouples the data and control plane from
every network element and moves the control plane to an SDN controller. This controller may
be deployed as a cluster for high availability and scalability, and, over time, east-west APIs will
emerge that enable multicontroller federation. One of the deployment models of SDN is one in
which the controller communicates with each network element via the OpenFlow protocol. By
using overlay technologies (such as VXLAN and NVGRE), network administrators can simplify
the mobility of VMs by running on top of any Layer 3 network topology. With SDN, network
intelligence and state are logically centralized, allowing the network to be abstracted from the
applications, and the configuration of the whole network can be done from one place, instead of
from each individual device. SDN promises the ability to leverage low-cost and commoditized
hardware and will be a key element of real-time infrastructure (RTI), which allows the network to
adapt itself according to application needs on-demand.
Selection Guidelines
When selecting among the different data center Ethernet switching fabric solutions, it's very
important to compare the various vendors' architectures and features, and align them with your
specific use cases:
■ Vendors' architectures:
■ One- or two-tier architecture: Either fully or partially meshed solutions, these are the two
most common physical architectures. Select a vendor based on needs such as scalability,
oversubscription rate, latency and automation, instead of the actual number of network
tiers. Prefer vendors that allow scaling of the switching fabric with minimum intervention,
while keeping the desired oversubscription rate and latency across the switching fabric.
■ From an operational perspective, some vendors have a solid solution for defining and
managing the whole fabric as a single entity — one that is self-learning and adaptive. Other
vendors have a box-by-box management approach and still require configuration changes
when fabric topology changes. All vendors somehow still require some sort of manual
command line interface (CLI) configuration; however, for operational simplification, select a
vendor that will help diminish this problem via the use of intelligent and self-learning
techniques.
■ Features and use cases: DCB, FCoE, VM awareness and extreme low latency are all features
that should be selected based on use cases and business needs. For example, if your
environment is highly virtualized and cloud-oriented, then an architecture that provides an ability
to be elastic, scalable and virtualization-aware would prevail over an architecture where every
nanosecond can make a difference with latency as the prime factor. For example, if you run a
highly virtualized enterprise data center, then live VM migration is an important function of data
center capacity management and for operations efficiency. In this case, the amount of manual
configuration of network policy to support the moving VM is a good business case for saving
operational labor costs.
Page 8 of 10 Gartner, Inc. | G00249268■ Interoperability: Ensure that your vendor also offers standard base interfaces and protocols to
interoperate with networks running on legacy protocols like STP.
Technology Providers
Networking vendors have different branded names for their data center switching fabric solutions.
The main ones, listed in alphabetical order, are:
■ Alcatel-Lucent: Alcatel-Lucent Mesh
■ Arista: No specific name
■ Avaya: Avaya Virtual Enterprise Network Architecture (VENA) Fabric Connect
■ Brocade: VCS Fabric
■ Cisco: Unified Fabric
■ Dell: No specific name
■ Enterasys: OneFabric
■ Extreme Networks: Open Fabric
■ Huawei: Cloud Fabric
■ HP: FlexFabric
■ IBM: No specific name
■ Juniper: QFabric System
Recommended Reading
Some documents may not be available as part of your current Gartner subscription.
"Magic Quadrant for Data Center Network Infrastructure"
"Eight Key Impacts on Your Data Center LAN Network"
"Ending the Confusion About Software-Defined Networking: A Taxonomy"
"Clearing the Confusion About Fabric-Based Infrastructure: A Taxonomy"
Gartner, Inc. | G00249268 Page 9 of 10GARTNER HEADQUARTERS
Corporate Headquarters
56 Top Gallant Road
Stamford, CT 06902-7700
USA
+1 203 964 0096
Regional Headquarters
AUSTRALIA
BRAZIL
JAPAN
UNITED KINGDOM
For a complete list of worldwide locations,
visit http://www.gartner.com/technology/about.jsp
© 2013 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. or its affiliates. This
publication may not be reproduced or distributed in any form without Gartner’s prior written permission. If you are authorized to access
this publication, your use of it is subject to the Usage Guidelines for Gartner Services posted on gartner.com. The information contained
in this publication has been obtained from sources believed to be reliable. Gartner disclaims all warranties as to the accuracy,
completeness or adequacy of such information and shall have no liability for errors, omissions or inadequacies in such information. This
publication consists of the opinions of Gartner’s research organization and should not be construed as statements of fact. The opinions
expressed herein are subject to change without notice. Although Gartner research may include a discussion of related legal issues,
Gartner does not provide legal advice or services and its research should not be construed or used as such. Gartner is a public company,
and its shareholders may include firms and funds that have financial interests in entities covered in Gartner research. Gartner’s Board of
Directors may include senior managers of these firms or funds. Gartner research is produced independently by its research organization
without input or influence from these firms, funds or their managers. For further information on the independence and integrity of Gartner
research, see “Guiding Principles on Independence and Objectivity.”
Page 10 of 10 Gartner, Inc. | G00249268You can also read
