Next Stop: Exascale - Argonne Leadership Computing ...
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
P R E PA R I N G F O R E X A S C A L E
Next Stop: Exascale
Designed in collaboration with Intel and Cray, the ALCF’s
exascale system, Aurora, will help ensure continued U.S.
leadership in high-end computing for scientific research.
With Aurora’s 2021 arrival drawing closer, the ALCF continued AURORA ARCHITECTURE
to ramp up its efforts to prepare for the exascale system. In Aurora will be based on Intel’s Xeon Scalable processors
2019, there was a flurry of activity that included a high-profile and high-performance Intel Xe GPU compute accelerators.
announcement event, the unveiling of new architectural The system will rely on Cray’s Shasta exascale-class
details, collaborations to further develop an ecosystem that architecture and Slingshot interconnect technology, which
will enable science in the exascale era, and several training can provide concurrent support for advanced simulation
offerings designed to prepare researchers for Aurora. and modeling, AI, and analytics workflows. Aurora will
leverage historical advances in software investments along
On March 18, leaders from the U.S. Department of Energy
with increased application portability via Intel’s oneAPI.
(DOE), Argonne National Laboratory, Intel, and Cray
The supercomputer will also introduce a new I/O system
(now part of Hewlett Packard Enterprise) came together to
called Distributed Asynchronous Object Storage (DAOS) to
officially announce that the laboratory would be home
meet the needs of new exascale workloads.
to one of the nation’s first exascale systems with the arrival
of Aurora in 2021. At the 2019 Supercomputing (SC19) conference in November,
Intel unveiled several new architectural details and tools
The event, hosted at Argonne, included remarks from former
that have allowed researchers to ramp up their efforts to
U.S. Secretary of Energy Rick Perry, Intel CEO Robert
prepare for Aurora.
Swan, Cray CEO Pete Ungaro, University of Chicago President
Robert Zimmer, U.S. Senator Dick Durbin, U.S. Representative The system’s compute nodes will feature two Intel Xeon
Dan Lipinski, and Argonne Director Paul Kearns, who Scalable processors (code-named “Sapphire Rapids”) and
joined together to underscore the significance of reaching six general-purpose GPUs based on Intel’s Xe architecture
exascale. (code-named “Ponte Vecchio”) and optimized for HPC and
AI workloads.
From mapping the human brain to designing new functional
materials to advancing our understanding of mysterious Intel also announced that Aurora will have over 10 petabytes
cosmological phenomena, the ALCF’s forthcoming machine of memory and over 230 petabytes of storage. It will
will enable researchers to pursue science and engineering leverage the Cray Slingshot fabric to connect nodes across
breakthroughs that were not possible with previous the massive system.
supercomputers.
22 A LC F. A N L .G OV
Aurora’s compute nodes will be equipped with two Intel Xeon Scalable processors and six general-purpose GPUs based on Intel’s Xe architecture.
Image: Intel Corporation
Sustained Performance Platform
≥1 Exaflop DP Cray Shasta
Delivery Aggregate System Memory
CY 2021 >10 PB
Compute Node GPU Architecture Network Switch
2 Intel Xeon scalable “Sapphire Xe arch-based “Ponte Vecchio” 25.6 Tb/s per switch, from 64–200
Rapids” processors; 6 Xe arch-based GPU; Tile-based chiplets, HBM stack, Gbs ports (25 GB/s per direction)
GPUs; Unified Memory Architecture; 8 Foveros 3D integration, 7nm
fabric endpoints High-Performance Storage
On-Node Interconnect ≥230 PB, ≥25 TB/s (DAOS)
Software Stack CPU-GPU: PCIe
Cray Shasta software stack + Intel GPU-GPU: Xe Link Programming Environment
enhancements + data and learning Intel oneAPI , MPI , Open MP, C /C++,
System Interconnect Fortran, SYCL /DPC++
Cabinets Cray Slingshot; Dragonfly topology
>100 with adaptive routing
ALCF 2019 ANNUAL REPORT 23
P R E PA R I N G F O R E X A S C A L E
Aurora, the ALCF’s future exascale system, will
help ensure continued U.S. leadership in high-end
computing for scientific research.
The ALCF’s Intel-Cray exascale system is scheduled to arrive in 2021.
In addition to the new architectural details, Intel launched compilers (e.g., Clang/LLVM) that are critical to developing
its one API initiative, which aims to provide a unified efficient and portable exascale applications.
programming model that simplifies development for diverse
The ALCF is partnering with Altair to leverage the company’s
workloads across different architectures.
PBS Professional software to create the scheduling system
for Aurora. This effort will include working with the PBS Pro
TESTING AND DEVELOPMENT
open source community to develop an effective solution
The initial oneAPI beta software release provided a new
for exascale job scheduling and workload management.
avenue for researchers to prepare for the Aurora
architecture. Available via the Intel DevCloud, users were EARLY SCIENCE
able to gain free access to compilers, libraries, and tools The ALCF’s Aurora Early Science Program (ESP) is designed
for testing and development work in preparation for to prepare key applications for the scale and architecture
the exascale system. of the exascale machine, and field-test compilers
Intel integrated GPUs, Gen9 and later, are also helping and other software to pave the way for other production
researchers gain a better understanding of Aurora’s Ponte applications to run on the system.
Vecchio GPU architecture. Available through Argonne’s Through open calls for proposals, the ESP has awarded
Joint Laboratory for System Evaluation (JLSE), Intel Gen9 pre-production computing time and resources to
integrated GPUs allow developers to test code performance five simulation projects, five data projects, and five learning
and functionality using programming models that will be projects. The diverse set of projects reflects the ALCF’s
supported on Aurora. effort to create an environment that supports emerging
data science and machine learning approaches alongside
EXASCALE PARTNERSHIPS traditional modeling and simulation-based research.
The ALCF is involved in numerous collaborative efforts to
develop and deploy tools and technologies that are By bringing together computational scientists, code
essential to enabling science in the exascale era. Staff developers, and computing hardware experts, the ESP
researchers are engaged in several activities within creates a collaborative environment for optimizing
DOE’s Exascale Computing Project (ECP), a multi-lab initiative applications and characterizing the behavior of the facility’s
dedicated to delivering a capable exascale computing future exascale system. In partnership with experts from
ecosystem for the nation. Argonne has strong presence on Intel and Cray, ALCF staff members are helping train
the ECP leadership team and in projects and working groups the ESP teams on the Aurora hardware design and how to
focused on application development, software technologies, develop code for it.
and hardware and integration.
ALCF staff members continue to contribute to the
advancement of software standards (e.g., C++, OpenMP),
programming models (e.g., oneAPI, SYCL, Kokkos), and
24 A LC F. A N L .G OV
The Aurora Programming Workshop welcomed ECP and ESP researchers
to the ALCF for guidance on preparing applications and software for the
exascale system.
USER TRAINING presentations on Intel’s Aurora programming models (e.g.,
In 2019, the Argonne-Intel Center of Excellence (COE) kicked OpenMP, SYCL/DPC++, OpenCL); open programming
off a series of intensive, hands-on sessions called models (e.g., Kokkos, Raja); development hardware; and
“hackathons” to help individual ESP teams advance efforts best practices and lessons learned thus far. Attendees
to port and optimize applications using the Aurora software also participated in hands-on sessions using the latest
development kit, early hardware, and other exascale Aurora software development kit.
programming tools.
In addition to on-site events, the ESP continues to offer
The six 2019 hackathons covered a wide range of application web-based tutorials to project teams on topics and tools
areas and featured a variety of activities tailored to relevant to leadership-scale computing resources,
each project, including the implementation of kernels with an emphasis on data-intensive and machine learning
and mini-apps using various Aurora programming models; subjects. The ECP and other DOE computing facilities also
testing on Intel Gen9 integrated GPUs serving as provide a multitude of workshops and webinars focused on
development platforms; targeted presentations and exascale tools and topics.
deep dives; and lengthy Q&A sessions with Intel
Together, all of these efforts are preparing the research
software and hardware experts.
community to harness the immense computing power of
COE staff coordinated with the ESP project teams before, Aurora and other future exascale systems to drive a new
during, and after the hackathons to prepare, facilitate, era of scientific discoveries and technological innovations.
and follow up on activities. The hackathons also provided
valuable input to Argonne and Intel on the merits
and deficiencies of various programming approaches, and
helped identify multiple bugs in the pre-alpha/alpha/beta
versions of the software development kit.
In April, the Argonne-Intel COE held a three-day workshop
to present a deep dive into the Aurora hardware and
software environment for all ESP teams. The event covered
a wide range of topics including the Aurora software stack;
its DAOS I/O system; data science and analytics workloads;
programming models; performance tools; and transitioning
project workflows from Theta to Aurora.
The ALCF hosted another exascale training event in
September—the Aurora Programming Workshop—for
both ESP and ECP research teams. The three-day workshop
was focused on preparing applications and software
technologies for Aurora. This event included
ALCF 2019 ANNUAL REPORT 25
P R E PA R I N G F O R E X A S C A L E
Simulation Projects
Readying Aurora for
Science on Day One
NWChemEx will provide the understanding needed to control molecular
processes underlying the production of biomass. Image: Thom H. Dunning
Jr., University of Washington and Pacific Northwest National Laboratory
Extending Moore’s Law Computing with Quantum
Monte Carlo
PI Anouar Benali
INST Argonne National Laboratory
High-Fidelity Simulation of Fusion Reactor Boundary
Plasmas
PI C.S. Chang
INST Princeton Plasma Physics Laboratory
NWChemEx: Tackling Chemical, Materials, and
Biochemical Challenges in the Exascale Era
PI Theresa Windus
INST Iowa State University and Ames Laboratory
Extreme-Scale Cosmological Hydrodynamics
PI Katrin Heitmann
INST Argonne National Laboratory
Extreme-Scale Unstructured Adaptive CFD
PI Ken Jansen
INST University of Colorado Boulder
26 A LC F. A N L .G OV
Data Projects Learning Projects
The catalysis project will combine data science techniques and quantum The PPPL team’s Fusion Recurrent Neural Network uses convolutional and
chemistry simulations to explore the otherwise intractable phase space resulting recurrent neural network components to integrate both spatial and temporal
from gas phase molecules on catalyst surfaces to find relevant configurations information for predicting disruptions in tokamak plasmas. Image: Julian
and the lowest transition states between them. Image: Eric Hermes, Sandia Kates-Harbeck, Harvard University; Eliot Feibush, Princeton Plasma Physics
National Laboratories Laboratory
Exascale Computational Catalysis Machine Learning for Lattice Quantum Chromodynamics
PI David Bross PI William Detmold
INST Argonne National Laboratory INST Massachusetts Institute of Technology
Dark Sky Mining Enabling Connectomics at Exascale to Facilitate
Discoveries in Neuroscience
PI Salman Habib
INST Argonne National Laboratory PI Nicola Ferrier
INST Argonne National Laboratory
Data Analytics and Machine Learning for Exascale
Computational Fluid Dynamics Many-Body Perturbation Theory Meets Machine
Learning to Discover Singlet Fission Materials
PI Ken Jansen
INST University of Colorado Boulder PI Noa Marom
INST Carnegie Mellon University
Simulating and Learning in the ATLAS Detector at
the Exascale Virtual Drug Response Prediction
PI Walter Hopkins* PI Rick Stevens
INST Argonne National Laboratory INST Argonne National Laboratory
*The original PI Jimmy Proudfoot has retired.
Accelerated Deep Learning Discovery in Fusion
Extreme-Scale In-Situ Visualization and Analysis of Energy Science
Fluid-Structure-Interaction Simulations
PI William Tang
PI Amanda Randles INST Princeton Plasma Physics Laboratory
INST Duke University and Oak Ridge National Laboratory
ALCF 2019 ANNUAL REPORT 27You can also read
