Emerging Technologies for HPC Storage - Dr. Wolfgang Mertz CTO EMEA Unstructured Data Solutions June 2018 - Dell EMC HPC Community
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Emerging Technologies for HPC Storage Dr. Wolfgang Mertz CTO EMEA Unstructured Data Solutions June 2018
The very definition of HPC is expanding
Blazing Fast Speed Accessibility and flexibility
Traditional High High Performance
Performance Data Analytics
Computing Complex or time-critical ‘big
Computationally-intensive data’ analytics workloads
modeling and simulation • Genomics
• Computer-aided engineering • Financial analytics
• Weather forecasting • Business intelligence
• Oil exploration
Artificial Intelligence
Machine and deep learning
applications
• Fraud / anomaly detection
• Predictive maintenance
2 © Copyright 2018 Dell EMC
• Personalized medicine
HPC vs HPDA from a data perspective
HPC
HPDA
Time
Start End
3 © Copyright 2018 Dell EMC
Value of data over time…
Value of
Data ($) “Big Data”
“Fast Data”
Time
µs ms s hour day month year yr+
4 © Copyright 2018 Dell EMC
HPC Storage Challenges
High Availability,
Backup, Data Data Sharing and Management &
Critical Challenge Performance Protection Accessibility Integration
Performance
Uptime important, but no
Traditional ‘/scratch’ bare metal, parallel access special backup
Not the critical feature Important, but not the
requirements critical feature
Persistence
Scalable performance, Able to fulfill compliance Pre and post Management
Traditional ‘/home’ tunable for workload requirements, protect processing, analytics, functionality and
important data desktop access support; connections
with other tools
5 © Copyright 2018 Dell EMCNVMe-oF
Dell EMC
&
rd
3 Party Job
Scheduler
Performance
HPC Fast Storage
(Lustre, BeeGFS, GPFS)
Isilon
HPC NFS Storage Elastic Cloud Storage
(ECS)
Virtustream
Scratch Project Archive
Capacity
6 © Copyright 2018 Dell EMCNVMe Usage Scenarios: Local Dedicated Devices
Up to 24 U.2 Up to 24 U.2 Up to 24 U.2 Up to 24 U.2
NVMe (R740XD) NVMe (R740XD) NVMe (R740XD) NVMe (R740XD)
Each host has
one or more
dedicated
NVMe devices
Targets: Up to 4 Intel AIC Targets: Up to 4 Intel AIC Targets: Up to 4 Intel AIC Targets: Up to 4 Intel AIC
NVMe P3700 2TB NVMe P3700 2TB NVMe P3700 2TB NVMe P3700 2TB
7 © Copyright 2018 Dell EMCNVMe Usage Scenarios: Sharing NVMe over Fabrics
Targets: Up to 24 U.2
NVMe (R740XD) Even hosts with no space or
Targets U.2 NVMe support for NVMe can use
NVMeF devices.
NVMeF Host
Client
NVMeF Host
Targets: Up to 4 Intel AIC
Client
NVMe P3700 2TB 100 Gb/s
Targets: Up to 4 Intel AIC Each host can mount one or
NVMe P3700 2TB more NVMe Targets.
8 © Copyright 2018 Dell EMCNVMe-oF Test System 1
R730 NVMeF Targets
Server
Mellanox EDR R730 NVMeF
E5-2690 v4 @ 2.60 GHz. Host Client 1
ConnectX-5
256 GiB DDR4 2166 MHz
R730 NVMeF
Host Client 2
100 Gb/s
2x Xeon E5-2690 v3 @ 2.60 GHz
256 GiB DDR4 2166 MHz
Targets: 4 Intel AIC Each host can mount one or more
NVMe P3700 2TB Targets.
RHEL 7.4 x86_64 (GA level), kernel version 3.10.0-693
Native Drivers
9 © Copyright 2018 Dell EMCNVMe-oF Test System 2
R730 NVMeF Targets
Server
R730 NVMeF
E5-2690 v4 @ 2.60 GHz. Host Client 1
256 GiB DDR4 2166 MHz OmniPath
R730 NVMeF
Host Client 2
100 Gb/s
2x Xeon E5-2690 v3 @ 2.60 GHz
256 GiB DDR4 2166 MHz
Targets: 4 Intel AIC Each host can mount one or more
NVMe P3700 2TB Targets.
RHEL 7.4 x86_64 (GA level), kernel version 3.10.0-693
Native Drivers
10 © Copyright 2018 Dell EMCConfiguration Details of NVMe-oF
• 3 Dell PowerEdge R730, one used as (target) server connected to two clients (hosts).
– Clients Dual Intel Xeon E5-2690 v3 @ 2.60 GHz.
– Server Dual Intel Xeon E5-2690 v4 @ 2.60 GHz.
– 256 GiB of DDR4 @ 2133 MHz.
• Omni-Path adapters installed on slot 4 (PCIe x16) connected to a switch.
• RHEL 7.4 x86_64 (GA level), kernel version 3.10.0-693
• 4 Intel P3700 2TB AIC adapters on the server slots 1, 2, 3 & 5 (all PCIe x8)
• FIO 2.99 compiled on each machine with libaio support
• DirectIO and no buffered IO was used to prevent RAM cache
• Ramp up time was one hour and test time was limited to 300 seconds for each data point
• Each write test with a different block size was followed by a consistent read test
11 © Copyright 2018 Dell EMCBandwidth Baseline
ib_write_bw -F -R -a 172.20.1.1
Infiniband EDR (100 Gb/s) BW Average OmniPath (100 Gb/s) BW Average
12000
10000
8000
MB/sec
6000
4000
2000
0
2 4 8 16 32 64 128 256 512 1K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M 8M
Package Size
12 © Copyright 2018 Dell EMCGoing to Next Level with Hardware – ME4
• RAID Array: 2u12, 2u24, 5u84
• Expansion: 2u12,2u24,5u84 (DAE)
• Backend Interface:12G SAS
• FE Interface:
• 16 FC 4 ports per controller
• 10G iSCSI 4 ports per controller (SFP+ or BaseT)
• 12G SAS 4 ports per controller
• Reads IOPS: 320K – 4x over MD3 ME4012: 12-drive RBOD
ME4024: 24-drive RBOD
• Seq. Reads: 7000 MB/s
ME4084: 84-drive RBOD
• Seq. Writes: 5500 MB/s – 2.6x over MD3 ME412: 12-drive Expansion
• Total system drive count: 336 -1.75x over MD3 ME424: 24-drive Expansion
• Raw Capacity: 4PB ME484: 84-drive Expansion
• Single or Dual Controller
Note: ME Expansion Units (DAE) cannot be connected
to a server directly (not a server-attached JBOD)
13 © Copyright 2018 Dell EMCParallel File System – Lustre w/ ME4
Dell PowerVault Dell PowerVault
ME4024 ME4024
(optional for DNE)
Dell PowerVault
ME4084
14 © Copyright 2018 Dell EMCDell EMC Isilon Scale-out NAS
High Performance to Archive • Few TBs to 100 PB in a single file system
– Up to 1.5 TB/sec Aggregate Read
• Policy-based Automatic Tiering
F-Series – Flash, SAS, Sata
Files
Tier 1
• Native Multi Protocol Access
Reduced cost/TB – NFS, CIFS, HDFS, Swift
• Enterprise Features for Data Management, Long Term
Archive and Compliance
H-Series
Tier 2
CloudPools
A-Series
Tier 3
15 © Copyright 2018 Dell EMCECS Scale-Out Object Store Modern archive Universal archive for existing primary storage. Replaces tape. No changes to applications or operations. Archive always online of analytics workflows Cloud native Enable new healthcare business operations. Cloud economics and ease of use on-premise. Lower TCO compared to public cloud providers Scalability Deployable in clusters for petabyte and exabyte scalability Data protection Provides geo-distributed data protection with no single point of failure. Globally accessible. One namespace. Multi-tenant architecture Accelerate cloud native applications Future healthcare IoT applications on private infrastructure Operational flexibility Multi-protocol support for legacy & modern applications 16 © Copyright 2018 Dell EMC
Emerging Technologies for Persistent Storage
• Higher Scale
– 100s PB for File
– Exabyte for Object
• High Performance Object
• Removing Protocol Overheads
• Gen 7
17 © Copyright 2018 Dell EMCYou can also read
