DMA ST 2 The Digital Scientific Method - M. Al-Turany (GSI) G. Juckland. (HZDR) - Indico
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
DMA ST 2 ─ The Digital Scientific Method
M. Al-Turany (GSI)
G. Juckland. (HZDR)
www.helmholtz.de
Milestones:
2023
• DMA repository of interconnectable, modular software in full
operation
2025
• Toolbox for near-realtime data analysis at extreme scales
available
2027
• Surrogate models of multi-source, multi-modal experiments
2
M. Al-Turany
Common Interest
§ Next generation computing for simulation & analysis
§ Complex data analysis & data fusion
§ Applications of Machine Learning
§ Knowledge Extraction & Data Reduction
§ High Throughput Transport
M. Al-Turany 3
Status
§ Up to now almost no comunicatin within ST2 partners
§ Very limited contact between GSI and HZDR
§ Will this change? Or everybody will do his daily
business und once/twice per year we claim that we
work together and write a report?
M. Al-Turany 4
Trying to collect a list of activities!
Last year we tried to get a list of
project names and responsible
persons from the different
institutes involved i.e: DESY, FZJ,
GSI, HZB, HZDR and HZG
M. Al-Turany 5
1. Next generation computing for simulation & analysis
§ FZJ (active)
§ Software development for the analysis of scattering Experiments
§ Products: https://jugit.fz-juelich.de/mlz
§ Contact: Joachim Wuttke
§ GSI (active)
§ Heterogeneous Computing for Reconstruction and Analysis of Experiment Data
§ Tools : FairRoot, FairMQ, Vc
§ Experiments: Alice, Ship, CBM, Panda, R3B
§ Contact Person(s): ): M. Al-Turany, M. Kretz
6
M. Al-Turany
1. Next generation computing for simulation & analysis
§ HZDR (active)
§ Single-source Heterogeneous Computing for Simulation & Data Analysis
§ Tools: Alpaka, MallocMC, llama
§ Experiments: CMS (CERN), DRACO/Penelope (HZDR), HIBEF (XFEL)
§ Contact Person(s): J. Stephan, R. Widera
§ DESY: (active)
§ Frank Gaede, Anton Barty
§ HZG (active)
§ Intensity-based µCT reconstruction techniques optimized for parallel processing (e.g. Alpaka)
§ HZB (Observer)
7
M. Al-Turany
2. Complex data analysis & data fusion
§ FZJ (active)
§ Software development for the analysis of scattering Experiments
§ Products: https://jugit.fz-juelich.de/mlz
§ Contact: Joachim Wuttke
§ GSI (active)
§ Online Data Reconstuction/Reduction of Particle Physics Experiments
§ Tools: FairRoot, FairMQ
§ Experiments: Alice, CBM, Panda
§ Contact Person(s): M. Al-Turany, D. Klein, A.Rybalchenko
8
M. Al-Turany
2. Complex data analysis & data fusion
§ HZDR(active)
§ In-situ scattering & radiation transport for plasma simulations
§ Tools: PIConGPU
§ Experiments: DRACO, Penelope, HIBEF
§ Contact Person(s): R. Widera, A. Debus
§ DESY: (active)
§ Christoph Wissing, Anton Barty.
§ HZB (Observer)
M. Al-Turany 9
3. Applications of Machine Learning
§ FZJ (active)
§ Machine learning for experiment control and data analysis
§ Contact: Marina Ganeva
§ GSI (observer)
§ HZB (Observer)
§ DESY (active)
§ Ruecker, Heuser
§ HZDR (active)
§ Surrogate Models for WDM, HED, Lab-Astro & Plasma-driven accelerators from multi-source,
multi-modal experimental setups
§ Helmholtz AI (Matter) Young Investigator Group
§ Contact Person: N. Hoffmann
10
M. Al-Turany4. Knowledge Extraction & Data Reduction
§ GSI (active)
§ Knowledge Extraction from Nuclear Physics Experiments
§ Tools: FairRoot, R3BRoot
§ Experiment: R3B
§ Contact Person(s): R. Karabowicz, D. Kresan
§ DESY: (active)
§ Frank Gaede, Anton Barty
§ HZG (Observer)
§ HZB (Observer)
11
M. Al-Turany4. Knowledge Extraction & Data Reduction
§ HZDR (active)
§ Knowledge Extraction from X-ray Scattering Images via Forward & Inverse Methods, Reduced
Image Representations
§ Experiments: HIBEF
§ Contact Person(s): H. Meissner
12
M. Al-Turany5. High Throughput Transport
§ GSI (active)
§ Stream Processing of Experiment and Simulation Data
§ Tools : FairMQ, ODC, DDS
§ Experiments: Alice, CBM, Panda
§ Contact Person(s): M. Al-Turany, D. Klein, A.Rybalchenko
§ HZDR (active)
§ Data packing, I/O staging
§ Tools: OpenPMD
§ Contact Person(s): R. Widera
13
M. Al-Turany5. High Throughput Transport
§ DESY (active)
§ Martin Gasthuber
§ HZG (Observer)
§ HZB (Observer)
§ FZJ (Observer)
14
M. Al-TuranyTrying to collect a list of activities! Results
Either we improve the comunication or
we agree that we do not need to
communicate!
15
02.02.2021 M. Al-TuranyThe Good News 02.02.2021 M. Al-Turany 16
From libVc to libstdc++
Ma
tth
§ stdx::simd out of the box on Linux ias
Kr
§ with GCC, Clang, and ICC (all use libstdc++ by default) etz
§ part of GCC 11
§ #include
§ https://godbolt.org/z/PKjWKf
§ 4 instructions, 1 vs. 16 results
Vec Vec
17
02.02.2021 M. Al-TuranyPIConGPU – A Trailblazer for Exascale Codes
The „Lama“-Tools from
HZDR/CASUS attract more
interest:
l CERN Hackathon in June 2020
l → Pickup in HEP Community
l Project migrated to own group
on GitHub:
https://github.com/alpaka-group
l More tomorrow...
Alpaka 0.6.0 now with
l Full support for HIP on AMD and NVIDIA GPUs
l Initial support for OpenMP target offloading
l Experimental support for OpenACC
18
02.02.2021 M. Al-Turany 18• Reduce data volumes by doing (quasi) online reconstruction
• Each and every event needs to be processed, no rejection
• High Throughput (and not Performance) Computing problem
2021 3.4 TB/s 90 GB/s (50 PB/yr)
1 TB/s (22 PB/yr)
2025 1 GB/s
2025 0.3 TB/s (24 PB/yr)
1 GB/s
02.02.2021 M. Al-Turany 19• Reduce data volumes by doing (quasi) online reconstruction
• Each and every event needs to be processed, no rejection
• High Throughput (and not Performance) Computing problem
2021 3.4 TB/s 90 GB/s (50 PB/yr)
1 TB/s (22 PB/yr)
2025 1 GB/s
2025 0.3 TB/s (24 PB/yr)
1 GB/s
02.02.2021 M. Al-Turany 20ALICE Upgrade
• continuous readout
2021 • x50 event rate
3.4 TB/s
50 kHz
Online/Offline
Facility
• Aim is to reduce data volume by doing (quasi)
online reconstruction
• Each and every event needs to be processed,
no rejection
• High Throughput (and not Performance) (50 PB/y) Storage 90 GB/s
Computing problem
02.02.2021 M. Al-Turany 21ALICE Upgrade
• continuous readout
2021 See • x50 event rate
Gen talk b
y
http eric to Andr 3.4 TB/s
s:// o ey
ind ls for Lebe
ico. h d
des igh th ev to
y.de rou mor 50 kHz
/ev ghp row
ent u :
/28 t onlin
053 e
/co proc Online/Offline
ntr ess Facility
ib i
• Aim is to reduce data volume by doing (quasi) u tion ng
online reconstruction s/9
784
• Each and every event needs to be processed, 0/
no rejection
• High Throughput (and not Performance) (50 PB/y) Storage 90 GB/s
Computing problem
02.02.2021 M. Al-Turany 22• Reduce data volumes by doing (quasi) online reconstruction
• Each and every event needs to be processed, no rejection
• High Throughput (and not Performance) Computing problem
2021 3.4 TB/s 90 GB/s (50 PB/yr)
1 TB/s (22 PB/yr)
2025 1 GB/s
2025 0.3 TB/s (24 PB/yr)
1 GB/s
02.02.2021 M. Al-Turany 23August 26th, 2020 mCBM@SIS18 2021/22 | Christian Sturm, GSI 24
The mCBM experiment - precursor and demonstrator for
CBM @ SIS100
DAQ container
green IT cube
input stage
optical
fibers
processing stage
300 m
optical fibers 50m
FLES Timeslice Building
triggerless-
streaming FEE 50 m
DPB: FPGA, Event
1m 300 m Reconstruction
assigning Copper
GBTx µSlice
optical optical
time stamps building, Green Event Selection
to hits (FLES IT
TFC interface) Cube ArchivingVirtual Cluster: Share hardware for online clusters
Detectors
Data base Triggers
(configuration) DAQ
ECS
Online Time slice
Cluster building nodes
Re-Use what we already developed
and tested for ALICE Run 3
ECS GUI/CLI
25
02.02.2021 M. Al-TuranyPossibility of cooperation with EOSC
2023
• DMA repository of interconnectable, modular software in full
operation
02/2019 - 08/2022
A sustainable open-access repository to share scientific software and services to the science
community and enable open science
26
M. Al-TuranyDiscussion
For sure there are some good news from other
institutes but due to the bad comunication I miss
them!!
How to proceed?
02.02.2021 M. Al-Turany 27Discussion:
§ We are all busy with our projects, beam times, dead lines etc.
§ We are all fed up with meetings
§ Claiming that we will simply use the same tools is not realstic
§ How to optimize our time and effort?
§ We are different communities supporting different experiments and tools; Some problems
are similar but many are different, how to identify common interest on technical level?
02.02.2021 M. Al-Turany 28You can also read
