EXPERIMENTING WITH VIRTUAL SDXS USING CHAMELEON AND EXOGENI - Paul Ruth RENCI - University of North Carolina - SILECS
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
www. chameleoncloud.org
EXPERIMENTING WITH VIRTUAL SDXS USING
CHAMELEON AND EXOGENI
Paul Ruth
RENCI – University of North Carolina
pruth@renci.org
A P RIL 6 , 2 0 1 8 1
OUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.org
OUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.org
NSF GLOBAL ENVIRONMENT FOR
NETWORK INNOVATIONS (GENI)
Virtual laboratory for networking and distributed systems research and education
www. chameleoncloud.org
GENI FEDERATION
Federated identity
InCommon
X.509 identity certificates
Common APIs
Aggregate Manager
Clearinghouse
Agreed upon resource description language
RSpec
ExoGENI translates relevant portions from NDL-OWL to RSpec and back as needed
Several major portions
ExoGENI, InstaGENI, WiMax, Internet2 AL2S, ESnet
Federation with EU FIRE effort
www. chameleoncloud.org
EXOGENI
Virtual Compute and Virtual Network Infrastructure
Storage Infrastructure
Network Provisioning APIs (DOE ESNet
Cloud APIs (Amazon EC2 ..) OSCARS, Internet2, OESS, OGF NSI …)
Cloud Providers Network Transit Providers
www. chameleoncloud.org
EXOGENI VM VM VM VM
VM VM VM VM
Workflows
Mutually Isolated Slices
of Virtual Resources
VM VM VM VM
VM VM VM VM
VM VM VM VM
VM VM VM VM
Edge Providers
(Compute Clouds and Network Providers)
Mutually Isolated
Virtual Networks
www. chameleoncloud.org
EXOGENI
Relationship to GENI
One of two computational testbeds built for GENI
Implements GENI API
Accepts GENI users
Notable features:
Wide scale footprint (20 sites)
Edge clouds (OpenStack)
Dynamic layer 2 circuits between sites
Stitchports: layer 2 connections to external resources
Limitations
Small scale computational sites
No core network control
www. chameleoncloud.org
EXOGENI TOPOLOGY
www. chameleoncloud.org
EXOGENI TOOLS
www. chameleoncloud.orgEXOGENI: STITCHING
www. chameleoncloud.orgOUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.orgCHAMELEON PHASE 1 IN A NUTSHELL
Deeply reconfigurable: “As close as possible to having it in your lab”
Deep reconfigurability (bare metal) and isolation
Power on/off, reboot from custom kernel, serial console access, etc.
But also – modest KVM cloud for ease of use
Large-scale: “Big Data, Big Compute research”
~650 nodes (~15,000 cores), 5 PB of storage distributed over 2 sites connected
with 100G network…
…and diverse:ARMs, Atoms, FPGAs, GPUs, etc.
Blueprint for a sustainable production testbed:“cost-effective to deploy, operate, and
enhance”
Powered by OpenStack with bare metal reconfiguration (Ironic)
Open production testbed for Computer Science Research
Project started in 10/2014, testbed available since 07/2015
Currently 1,600+ users, 300+ projects
www. chameleoncloud.orgCHAMELEON: PHASE 1 HARDWARE To UTSA, GENI, Future Partners
Switch
Standard Core Services
Cloud Unit Front End and Data
Mover Nodes 504 x86 Compute Servers
42 compute 48 Dist. Storage Servers
4 storage 102 Heterogeneous Servers
x2 16 Mgt and Storage Nodes
Chameleon Core Network Chicago
SCUs connect to 100Gbps uplink public network Austin
core and fully (each site)
connected to
each other
Heterogeneous
Switch Cloud Units
ARMs, Atoms, low
Standard Core Services power Xeions, FPGAs,
Cloud Unit 3.6 PB Central File GPUs, SSDs, etc.
42 compute Systems, Front End
4 storage and Data Movers
x10
www. chameleoncloud.orgNEW HARDWARE
4 new Standard Cloud Units (32 node racks in 2U chassis)
3x Intel Xeon “Sky Lake” racks (2x @UC, 1x @TACC)
1x future Intel Xeon rack (@TACC) in Y2
Corsa DP2000 series switches
2x DP2400 with 100Gbps uplinks (@UC)
1x DP2200 with 100Gbps uplink (@TACC)
Each switch will have a 10 Gbps connection to nodes in the SCU
Optional Ethernet connection in both racks
More storage configurations
Global store @UC: 5 servers with 12x10TB disks each
Additional storage @TACC: 150 TB of NVMes
Accelerators: 16 nodes with 2 Volta GPUs (8@UC, 8@TACC)
Maintenance, support and reserve
www. chameleoncloud.orgNEW HARDWARE
4 new Standard Cloud Units (32 node racks in 2U chassis)
3x Intel Xeon “Sky Lake” racks (2x @UC, 1x @TACC)
1x future Intel Xeon rack (@TACC) in Y2
Corsa DP2000 series switches
2x DP2400 with 100Gbps uplinks (@UC)
1x DP2200 with 100Gbps uplink (@TACC)
Each switch will have a 10 Gbps connection to nodes in the SCU
Optional Ethernet connection in both racks
More storage configurations
Global store @UC: 5 servers with 12x10TB disks each
Additional storage @TACC: 150 TB of NVMes
Accelerators: 16 nodes with 2 Volta GPUs (8@UC, 8@TACC)
Maintenance, support and reserve
www. chameleoncloud.orgCORSA DP2000 SERIES SWITCHES
Hardware Network Isolation
Sliceable Network Hardware
Tenant controlled Virtual Forwarding Contexts (VFC)
Software Defined Networking (SDN)
OpenFlow v1.3
User defined controllers
Performance
10 Gbps within a site
100 Gbps between UC/TACC (Aggregated)
www. chameleoncloud.orgNETWORK HARDWARE
Internet 2 AL2S, GENI, Future Partners
Chameleon Core Network
100Gbps uplink public network
100 Gbps
(each site)
(Aggregate)
Stacked Switches 100 Gbps
(Logically One) (Aggregate)
Corsa DP2400 Corsa DP2400 Corsa DP2200
Standard Cloud Standard Cloud Standard Cloud
Unit Unit Unit
Chicago
Austin
www. chameleoncloud.orgISOLATED VIRTUAL SDN SWITCH
Isolated Tenant Networks
Corsa Switch
BYOC– Bring your own
controller: isolated user
controlled virtual OpenFlow VFC
VFC
switches (coming soon) (Tenant A) (Tenant B)
Compute Compute Compute Compute
Node Node Node Node
(Tenant A) (Tenant A) (Tenant B) (Tenant B)
OpenFlow Ryu
OpenFlow
Controller Controller
(Tenant A) (Tenant B)
Standard Cloud Unit
www. chameleoncloud.orgCHAMELEON: SDN EXPERIMENTS
Internet 2 AL2S, GENI, Future Partners
Chameleon Networking
Austin
RENCI added to the team Chameleon Core Network
Hardware Network Isolation 100Gbps uplink public network
Corsa DP2000 series Chicago
OpenFlow v1.3 Corsa
Sliceable Network Hardware DP2400
Switch VFC VFC
Tenant controlled Virtual Forwarding
Contexts (VFC) (Tenant A) (Tenant b)
Isolated Tenant Networks
BYOC – Bring your own controller Compute Compute Compute Compute
Node Node Node Node
Wide-area Stitching
(Tenant A) (Tenant A) (Tenant B) (Tenant B)
Between Chameleon Sites (100 Gbps) Ryu
OpenFlow OpenFlow
ExoGENI Controller Controller
Campus networks (ScienceDMZs) (Tenant A) (Tenant B)
Standard Cloud Unit
www. chameleoncloud.orgOUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.orgEXOGENI: INTER-SLICE STITCHING
www. chameleoncloud.orgEXOGENI: INTER-SLICE STITCHING
www. chameleoncloud.orgEXOGENI: INTER-SLICE STITCHING
Public Internet
www. chameleoncloud.orgEXOGENI: INTER-SLICE STITCHING
Starligh
t
www. chameleoncloud.orgEXOGENI: INTER-SLICE STITCHING
Service Slice Client Slice
Starligh
t
www. chameleoncloud.orgOUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.orgEXOGENI TO CAMPUS STITCHING
Control plane APIs
e.g. ORCA e.g. Plexus e.g. OSCARS e.g. GENI-API
L2 egress BEN
IP egress I2/A2LS
GENI
ESnet
IP core
Phys (L3) Circuit fabric Other GENI sites
providers Other campuses
DMZ Other facilities
DTN SDSN
Duke campus boundary
Duke University Software Defined Science Network (SDSN)
Science DMZ
www. chameleoncloud.orgEXOGENI TO CAMPUS STITCHING
Stitchport: Named meeting point linking
a layer 2 circuit between ExoGENI and
external resources.
Stitchport Duke SDSN
www. chameleoncloud.orgMULTI-TESTBED EXPERIMENTS
Service Slice Client Slice
Starligh
t
Client Campus
www. chameleoncloud.orgOUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.orgCHAMELEON TO EXOGENI STITCHING
• ExoGENI slice
• Dynamic Chameleon
Stitchport
Stitched L2 path
• Dynamic VLANs
• Connectivity to
ExoGENI Stitchport
www. chameleoncloud.orgCHAMELEON TO EXOGENI STITCHING
Stitching between ExoGENI and Chameleon nodes
StitchPort
www. chameleoncloud.orgINTER-TESTBED EXPERIMENTS
Service Slice Client Slice
Starligh
t
Client Campus
www. chameleoncloud.orgINTER-TESTBED EXPERIMENTS
Service Slice Client Slice
Starligh
t
Client Campus
Client Slice
www. chameleoncloud.orgINTER-TESTBED EXPERIMENTS
Service Slice Client Slice
Starligh
t
Client Campus
Client Slice
www. chameleoncloud.orgINTER-TESTBED EXPERIMENTS
Virtual SDX
Service Slice Client Slice
Starligh
t
Client Campus
Client Slice
www. chameleoncloud.orgOUTLINE
Background
ExoGENI testbed (wide footprint edge cloud)
NSF Cloud Chameleon testbed (mid-scale cloud)
Experiments Spanning Testbeds
Inter-slice stitching
Campus stitching
Inter-testbed stitching
Software Defined eXchange (SDX) Experiments
SAFE SDX (RENCI, DUKE, US DOE/Esnet)
SciDAS (Clemson, RENCI, Washington State University)
www. chameleoncloud.orgCICI SAFE PROJECT
“Creating Dynamic Superfacilities the SAFE Way”
Paul Ruth, Cong Wang, Mert Cevik, RENCI
Jeff Chase, Yuanjun Yao, Qiang Cao, Victor Orlikowski. Charley Kneifel, Duke Univeristy
Nick Buraglio, ESnet
NSF CICI Award #1642142
www. chameleoncloud.orgSUPERFACILITY
Definition
Two or more existing facilities (e.g. instruments, compute resources, data repositories) using
high-performance networks and data management software in order to increase scientific
output.
Currently manually created
Superfacilities are purpose-built manually for a specific scientific application or community.
Trust: “handshake model”
Ideally automated
Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical
building blocks to construct dynamic superfacilities on demand.
www. chameleoncloud.orgSUPERFACILITY
Definition
Two or more existing facilities (e.g. instruments, compute resources, data repositories) using
high-performance networks and data management software in order to increase scientific
output.
Currently manually created
Superfacilities are purpose-built manually for a specific scientific application or community.
Trust: “handshake model”
Ideally automated
Advanced Science DMZs and federated Infrastructure-as-a-Service provide the technical
building blocks to construct dynamic superfacilities on demand.
Trust also needs to be automated
www. chameleoncloud.orgSUPERFACILITIES THE SAFE WAY
Duke Science DMZ ExoGENI Slice Other Campus
DTN
IDS IDS IDS IDS
Virtual SDX
• Automating Superfacilites • SAFE: Secure Authorization for
– Multiple domains Federated Environments
– Friction free L2 paths – Isolates applications from logic concerns
• Naked L2 paths are not secure • Certificate discovery (DAGs)
– Handshake model of trust is not possible • Logic inference
• Cryptography
• Virtual SDX (vSDX) – Logic scripting language
– Distributed • Slang (SAFE Language)
– Enforces SDX connectivity policy • Based on Datalog
– Enforces client’s forwarding policy – Shared certificate repository
(security, BGP, etc.) • Stores statements and DAGs
– Intrusion Detection System (Bro)
www. chameleoncloud.orgSCIDAS
Cost-Aware
Requester Optimize
iRODS PerfSONAR
Orchestrator Shim (aaS) Shim (aaS)
PerfSONAR
API API mapping
Network
1PB Stge/ FIONA 1PB Stge./FIONA 1PB Stge./FIONA
www. chameleoncloud.orgSCIDAS
Automated vSDX superfacility
www. chameleoncloud.orgTHANK YOU
pruth@renci.org
www. chameleoncloud.orgYou can also read
