Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...

Page created by Jerome Blair
 
CONTINUE READING
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
Progetti Europei: le proposte presentate nelle call
 EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche
         accenno ai proposal in altre call

                    Luciano Gaido
          Riunione CCR, Torino, 3 Aprile 2017
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
Ringraziamenti

Doverosi per:
• Chi ha contribuito alla preparazione delle proposte presentate
  per le call che si sono chiuse il 29 marzo scorso:
   – Davide Salomoni, Giacinto Donvito, Daniele Cesini, Cristina
     Duma, Alessandro Costantini, Tommaso Boccali, Daniele
     Spiga, Claudio Grandi, Marco Verlato, Roberto Barbera

• Chi ha fornito slide per questa presentazione:
   – Cristina Duma, Daniele Cesini, Davide Salomoni, Giacinto
     Donvito, Marco Verlato, Tiziana Ferrari

                                                                2
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
Sommario

   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-HUB (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       3
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-hub (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       4
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
Il punto di partenza: premessa
Il programma della Commissione Europea H2020 (2014-2020) è
strutturato in 3 distinti Work Programme:
• WP 2014-2015
     – Concluso: i progetti sono in corso, vedi prossima slide
• WP 2016-2017
     – In corso: per ICT, l’INFN è coinvolto in varie proposte già
       presentate o in corso di elaborazione e in un progetto già
       approvato
• WP 2018-2020
     – il draft del programma è stato definito, siamo nella fase di
       consultazione degli esperti nazionali

                                                                      5
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
Il punto di partenza
Progetti in corso in nel Work Programme 2014-2015:
• West-life:
   – Coordinatore: STFC, durata: 1.11.2015 – 30.10.2018, resp.
     naz. M. Verlato, Call: EINFRA-9-2015, budget totale: 4 M€,
     budget INFN: 344 K€
• EGI_Engage:
   – Coordinatore: EGI Foundation (EGI.eu), durata: 1.3.2015 –
     31.8.2017, resp. naz.: L. Gaido, Call: EINFRA-1-2014,
     budget totale: 8 M€,
       budget INFN: 599.5 K€
• INDIGO-DataCloud:
   – Coordinatore: INFN, durata: 1.4.2015 – 30.9. 2017, PI: D.
     Salomoni, Call: EINFRA-1-2014, budget totale: 11.14 M€,
     budget INFN: 2.08 M€
                                                                  6
Progetti Europei: le proposte presentate nelle call EINFRA-12, EINFRA-21 e INFRAIA-02 e qualche accenno ai proposal in altre call - Luciano Gaido ...
   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-hub (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       7
EOSCpilot

Titolo: The European Open Science Cloud for Research Pilot
Project
    • Call: INFRADEV-04-2016 (European Open Science Cloud for
      Research)
    • Budget totale richiesto: 9.8 M€
    • Coordinatore: STFC
    • Partecipanti: 33 beneficiary e 15 linked third parties
    • Responsabile nazionale: C. Duma
    • Durata: 24 mesi, iniziato il 1 gennaio 2017
    • Budget per INFN: 254.4 K€
    • Sezioni INFN coinvolte: CNAF, PG
Prossime slide: grazie a C. Duma
                                                            8
EOSCpilot objectives
The EOSCpilot represents a first step towards the development of
the European Open Science Cloud.
It will:
• Design and trial a stakeholder-driven governance framework
• Contribute to the development of European open science
   policy and best practice;
• Develop demonstrators of integrated services and
   infrastructures in a number of scientific domains, showcasing
   interoperability and its benefits;
• Engage with a broad range of stakeholders, crossing borders
   and communities, to build trust and skills

                                                               9
EOSCpilot WP structure

                         10
EOSCpilot expected output
• governance framework (rules of engagement, business plan, ...),

• 15 science demonstrators

• federated service management framework

• interoperability demonstrators

• identification of training needs

• open innovation platform for industry engagement

• workshops for engagement of different stakeholders

                                                              11
EOSCpilot science demonstrators
5 already selected:
• PanCancer Analysis of Whole genomes (EMBL): Sensitive genomic
   data for cancer patient health care
• ENVRI Radiative Forcing Integration (ICOS ERIC + ACTRIS + DKRZ +
   IPSL): Integration of heterogeneous climate data sources
• Research with Photons & Neutrons (DESY, ESFR, XFEL, ESS, EMBL,
   ILL): Exploitation of data from analytical facilities
• WLCG (CERN): Large-scale long-term data preservation and reuse
   of physics data
• TEXTCROWD (Univ. of Florence): Collaborative semantic
   enrichment of text-based datasets –TEXTCROWD

10 more demonstrators being selected through an open call

                                                             12
EOSCpilot Interoperability WP (WP6)
Objectives:
   • Research and Data Interoperability: data & services to be findable,
     accessible, interoperable and reusable (FAIR)
   • Infrastructure interoperability: facilitate the most adequate infrastructures
     for the treatment of extensive amounts of data. Demonstrate with multi-
     infrastructure, multi-community pilots
Key Output:
   • The design of a future EOSC based on federated interoperable services
     meeting the needs of the thematic research domains and wider user base
• Tasks:
   • T6.1: gap analysis & interoperability architecture [CNRS lead]
   • T6.2: EOSC Research and Data interoperability [ELIXIR lead]
   • T6.3: Interoperability pilots [INFN lead]

                                                                             13
EOSCpilot T6.3 - Interoperability pilots
Validation regarding:
   • AAI requirements for both e-infrastructures and scientific communities
     solutions offered by INDIGO- DataCloud, ELIXIR AAI, EUDAT B2ACCESS and
     AARC
   • Resource brokering solutions - spanning multiple infrastructures and user
     communities and aimed for high-level resource discoverability and
     addressability
   • Accessibility - EOSC local, Grid, HPC and Cloud resources accessible by
     multiple communities
   • Data accessibility – through personal resources, scientific portals, CLI
   • Interoperability - underlying distributed storage systems with the EOSC
     platform services; services and tools - as those provided by the EUDAT
     service suite and INDIGO-DataCloud toolbox

                                                                                14
   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-hub (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       15
EOSC-hub
Titolo: Integrating and managing services for the European Open
Science Cloud
    • Call: EINFRA-12-2017 (Data and Distributed Computing e-
      infrastructures for Open Science), topic a: Secure and agile
      data and distributed computing e-infrastructure
    • Budget totale richiesto: 30 M€
    • Coordinatore: EGI Foundation (EGI.eu)
    • Partecipanti: 74 beneficiari (più 20 linked third-parties) da
      36 paesi diversi
    • Responsabile nazionale: L. Gaido
    • Durata: 36 mesi
    • Budget per INFN: 1.8 M€ (al secondo posto dopo EGI.eu)
    • Sezioni INFN coinvolte: BA, CNAF, CT, PD, PG, TO              16
EOSC-hub
Proposal congiunto tra EGI, EUDAT e INDIGO-DataCloud per la call
EINFRA-12, topic a:
(a) Secure and agile data and distributed computing e-infrastructures:
fostering the integration of a secure, permanent, on-demand service-driven,
privacy-compliant and sustainable e-infrastructure incorporating distributed
databases, computing resources and software.

The challenge is to integrate at European level the geographically and
disciplinary dispersed resources to achieve economies of scale and efficiency
gains in providing the best data and computing capacity and services to the
research and education communities.
This action is interrelated to INFRADEV-04-2016, “European Open Science Cloud
for Research”.

                                                                               17
EOSC-hub

Oltre a background e strutture di gestione diverse, c’è un’altra differenza
sostanziale: EGI e EUDAT sono (o sono considerate) infrastrutture mentre
INDIGO-DataCloud è un progetto di sviluppo software                           18
EOSC-hub

Scultura di Georges Faures, attualmente sulla terrazza dell’hotel Bologna, in corso
Vittorio Emanuele, angolo via XX settembre
                                                                                      19
Objectives

• Simplify access to a broad portfolio of products, resources
  and service provided by major pan-European and
  international organizations through an open service catalogue
   – A cornerstone of the EOSC
• Remove fragmentation of service provisioning and access to
  digital services in Europe and beyond
   – Technical integration between common & thematic
     services
   – Service innovation, procurement, provisioning and access
• Increase innovation capacity of digital Infrastructures

                                                              20
Objectives (cont)

• Consolidate digital infrastructures by
   – Expanding capacities and capabilities
   – Improving discoverability, access, interoperability and
     sharing , across research communities and countries
• Extend access to integrated compute, storage, data and
  software to new user groups including high-education and
  industry, increase the user base
• Expand human capacity
   – Consolidate/Expand a distributed network of experts and
     service operators at local/national level

                                                               21
Service architecture

                              Community Support services
  Thematic        Thematic      Thematic       Thematic       Thematic   Thematic
    Service        Service       Service         Service       Service    Service

                                                                            Open
Federation                       Added Value Services                      Collab.
 Services                         Compute, Data, Software
                                Management and Preservation
                                                                          Platforms

     AAI,                                                                 Application
 Accounting,                                                              Repository,
 Monitoring,                      Basic Infrastructure                   Configuration
  Operations,                       Compute and Storage                  Management,
Security Coord.                                                          Marketplace
                                   Common services
Technology Readiness Level
Development           Drawing             Proof-of-
                                                                  Pilot
                                                                                       Pre-
                                                                                                           Production
Phase                  board              Concept                                   production

TRL         1                     2   3                4    5              6   7                 7    8                 9

                             Go/NoGo              Go/NoGo                 Go/NoGo           Go/NoGo

Service                                Define Scope             Roadmap            Roadmap                 Update
Definition          Basic Ideas
                                      Basic Roadmap                RC                 SR                  Roadmap

Release naming
Convention                                 Alfa                   Beta                 RC                   Stable

Service Portfolio
                                                      Portfolio                                  Catalog
Management

Documentation                                                     Basic        First version               Published

Operated by                                    Service Building                             Operations
                                                                                                                  23
Project structure
       S

                    24
Effort distribution
                                  S
Effort per Category of services               Total    %

Management and Coordination (WP1)             267     7%

Service Planning (WPs 2-3-12)                 348     9%

Service integration and delivery (WPs 4-10)   763     20%

Service Catalogue (WPs 5-6-7)                 1,001   26%

Service Adoption (WPs 8-9-11-13)              1,506   39%

Grand Total                                   3,885   100%

                                                             25
Engagement with research communities

• Thematic Service Providers
   – Interested in providing a thematic production service as part
     of e-Infra-12 and the future EOSC
• Early adopters
   – Interested in piloting common services, using and advancing
     cross-infrastructure usage for the benefit of their research
     community and beyond
   – Organized through “competence centers” bringing together
     e-Infrastructure providers and relevant research
     organizations and technology providers/experts

Both have already been selected via an open call, but an
additional call is expected during the project (about 0.5 M€
earmarked)                                                     26
Thematic Services
                             S
60 proposals received, 9 selected:
• CLARIN (European Research Infrastructure for Language
   Resources and Technology)
• DODAS (CMS)  Lead: INFN (vedi present. di D. Spiga)
• ECAS (climate studies)
• GEOSS (Global Earth Observation System of Systems)
• OPENCoasts (On-demand Operational Coastal Circulation
   Forecast Service)
• WeNMR (Worldwide e-Infrastructure for Nuclear Magnetic
   Resonance and structural biology)
• DARIAH (Digital Research Infrastructure for the Arts and
   Humanities)
• LifeWatch
• EO Pillar (Earth Observation)
                                                             27
Early Adopters/Competence Centers
The Competence Centre is driven by well established and mature
research infrastructure or international collaboration requiring
advanced and integrated data and computing services.
In the Competence Centre early adopters test, adapt, and
integrate the digital capabilities they need to pursue they
research, with the support of e-Infrastructure and technology
experts.
The Competence Centre will:
    • Run Proof of Concepts
    • Conduct Pilots
    • Prepare the production environment
    • Define appropriate business models to sustain the solutions
      after the end of the project
                                                             28
Competence Centers

Out of 51 proposals received, 7 Competence Centres have been
selected:
    • Elixir  Lead Partner: EMBL-EBI
    • Fusion  Lead Partner: CCFE
    • Marine  Lead Partner: IFREMER
    • EISCAT_3D  Lead Partner: EISCAT
    • EPOS-ORFEUS  Lead Partner: SURFsara
    • Radio Astronomy Competence Center (RACC) Lead
      Partner: ASTRON
    • ICOS  Lead Partner: SNIC
    • Disaster Mitigation Competence Centre Plus (DMCC+) 
      Lead Partner: ASGC, unfunded
                                                         29
Involvement of Industries

Organized through Business Pilots:
• relevant to EGI participants with a national business
   programme and have activities of pan-European
   impact/interest
• designed to foster innovation between e-Infrastructures and
   the private sector through building an ecosystem of SMEs, large
   industries, startups, researchers, accelerators, and investors to
   become active business partners of e-Infrastructures as
   customers and/or service providers. These initial pilots will
   serve as early demonstrators of the project’s Joint Digital
   Innovation Hub (DIH).
Selected via an open call at proposal preparation
                                                               30
Business Pilots

6 proposals selected, out of 31 received:
    • CyberHAB (Water body management sector)
    • Sports Smart Video Analysis (Sports sector)
    • Bot Mitigation Engine (Business sector)
    • ACTION Seaport (Local coastal authorities)
    • Space Weather Data Services for the future DRACO
      Observatory (Climate sector)
    • Furniture Enterprise Analytics - DataFurn (Furniture industry
      sector)
Two transversal activities will support these pilots, an OpenLab
and the commercialization support.

                                                               31
Linking programme

Goal:
• Linking of the EOSC-hub with local/national e-Infrastructures
  from all European regions to:
   – Contribute to user community engagement through the
      provisioning of infrastructure services to user communities
      of the EOSC-hub engagement roadmap, and to participate to
      direction giving activities involving EOSC-hub stakeholders.
   – Publish local/national services in the EOSC-hub catalogue in
      compliance to the EOSC-hub rules of engagement for service
      providers
Budget:
• about 500K€ in task T4.1 (4 to 6 PMs)
                                                              32
Ruolo dell’INFN/1

• WP1 (6 PM) per coordinamento WP10

• WP2 (24 PM) per attività di governance, strategy, service
  portfolio management

• WP4 (16 PM) per contributo a operations coordination (V.
  Spinoso)

• WP6 (45 PM) per maintenance di alcune componenti grid
  (CREAM, BDII, ARGUS, VOMS) e INDIGO (IAM, PaaS, FG, ….)

                                                              33
Ruolo dell’INFN/2

• WP7 (36 PM) per attività in 2 Thematic Service (DODAS e
  WENMR)

• WP10 (56 PM) per coordinamento del Technology Committee,
  coord. task T10.1 (technical roadmap), evoluzione service
  catalogue e supporto in varie aree tematiche (AAI, PaaS, User
  Interfaces, Data Solutions)

• WP11 (17 PM) per contributo ad attività di training e
  coordinamento task T6.4

• WP13 (51 PM) per provisioning di risorse per vari servizi 
  fondi attraverso meccanismo del Virtual Access              34
WeNMR Thematic Service/1
• Inheritance of EGI-Engage MoBrain Competence Centre
• Involved partners are the core partners of four past and current
  EU projects supporting the Structural Biology community:

 Participants:
     • University of Utrecht (coordinator)
     • Magnetic Resonance Center (CERM)/University of Florence
     • INFN-Padova
     • … and have also been partners in INDIGO-DataCloud,
       providing use cases to drive the development of novel cloud
       solutions                                                35
WeNMR Thematic Service/2

• 8 grid-enabled application web portals already in production
  (TRL9): 6 hosted at University of Utrecht (DISVIS, POWERFIT, HADDOCK,
  GROMACS, CS-ROSETTA, UNIO) and 2 at CERM (FANTEN, AMPS-NMR)
• ~ 20M normalized CPU-hours/year on EGI HTC platform (SLA in
  place with EGI)
• DISVIS, POWERFIT and AMPS-NMR pioneering the use of HTC
  GPGPU resources
 o via udocker tool (INDIGO devel.) and GPU-enabled CREAM-CE
   (INFN-Padova/Milano devel.)
• …but also exploring the way to move the workload to cloud (e.g. in
  2 INDIGO use-cases)

                                                                          36
WeNMR Thematic Service/3
• Planned activities encompass:
 • User support and training, Outreach and dissemination
 • Continuous operation of the various grid- and cloud-enabled web portals,
   their Consolidation / upgrading and provisioning
 • Consolidation of the job submission frameworks of the various portals
   building on DIRAC4EGI and INDIGO solutions (e.g. phasing out glite-WMS in
   favor of DIRAC or PaaS Orchestrator)
 • Integration of distributed data storage solutions (from EUDAT and OneData)
 • Implementation of AAI solutions developed by EGI and INDIGO
• INFN main role:
 •   Maintenance, operations and support of enmr.eu VO related services and
     users (VOMS servers, LFC, HTC and FedCloud resources, GPU resources)
 •   Supporting the service evolution by integrating (where needed) advanced
     INDIGO solutions like PaaS Orchestrator, IAM, OneData. E.g. INFN is already
     providing testing OneData storage to WeNMR users
                                                                             37
DODAS Thematic Service

Vedi presentazione di Daniele Spiga

                                      38
Sommario

   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-HUB (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       39
DEEP-HybridDataCloud
Titolo: Designing and Enabling E-Infrastructures for intensive
Processing in a Hybrid DataCloud
    • Call: EINFRA-21-2017, topic b: Research and Innovation
      Actions for e-Infrastructure prototypes, subtopic 2:
      Computing e-infrastructure with extreme large datasets
    • Budget totale richiesto: 2.99 M€
    • Coordinatore: CSIC
    • Partecipanti: 10
    • Responsabile nazionale: G. Donvito
    • Durata: 30 mesi
    • Budget per INFN: 375 K€, al secondo posto dopo CSIC
    • Sezioni INFN coinvolte: Bari, CNAF, Torino
Prossime slide: grazie a G. Donvito                              40
DEEP-HybridDataCloud

                       Siamo il secondo partner
                       insieme a LIP dopo il
                       coordinatore per numero di
                       PM e il secondo partner per
                       finanziamento in €

                                          41
DEEP-HybridDataCloud: project description

    Figure 3: Initial architecture proposed to support a deep learning application using
                                    private cloud services.

escribed use case represents one of the challenges, although a very relevant one, that the project
. Furthermore, the performance of this use case can be improved by accessing specialized
 ources to perform advanced and sophisticated analysis. These resources, in many cases are not
 he private cloud where the experiment takes place and therefore, could be provided by external
ng a profitable use of the new Hybrid Cloud functionalities provided by the DEEP Hybrid
roject.
ese requirements, the initial design of the architecture is extended to a Hybrid Cloud solution,
heme is proposed (Figure 4).

              Figure 4: Initial architecture using a Hybrid Cloud approach.
ocess applies to the implementation of the two-other generic computing intensive techniques
 ered in the pilot cases. The details of the research use cases are presented in section 2, but the
ecture being considered is presented in the figures below.
esign for the architecture to support massive analysis/post-processing of very large data sets in     42
s (see also Section 1.3.3) is presented Figure 5. Notice the possibility of integrating HPC
DEEP-HybridDataCloud: obiettivi
• Objective 1: Focus the interest of intensive computing
  techniques on the analysis of very large datasets, considering
  demanding cases from different research communities, in the
  context of the future generations of e-infrastructure.
• Objective 2: Evolve, up to production level, intensive
  computing services exploiting specialized hardware
  components, like GPUs, low-latency interconnects, and others
  usually accessed as “bare metal” resources. The services, based
  on open source software, will follow existing standards to
  guarantee their deployment and orchestration on different
  platforms.
• Objective 3: Integrate the intensive computing services under a
  Hybrid Cloud approach, assuring interoperability with the
  existing EOSC platforms and their services.                    43
DEEP-HybridDataCloud: obiettivi (cont)
• Objective 4: Define a “DEEP as a Service” solution to offer an
   easy integration path to the developers of final applications.
• Objective 5: Analyse the complementarity with other ongoing
   projects targeting added value services for the cloud, in
   particular those related to the management of extremely large
   datasets.
• Global Objective: Promote the use of intensive computing
   services by different research communities and areas, and their
   support by the corresponding e-infrastructure providers and
   open source projects.
Areas for dissemination via pilot applications: Medical Images,
Citizen Science, Earth Observation, Post-processing very large
datasets coming from large scale simulations in Lattice QCD,
Genomics, RadioAstronomy
                                                                 44
INFN in DEEP-HybridDataCloud
• Sezioni INFN coinvolte: Bari, CNAF e Torino
• Persone coinvolte nel proposal: Giacinto Donvito, Alessandro
  Costantini, Stefano Bagnasco.
• Attività dell’INFN:
   – Use Case: INFN team in Bari collaborates in the application of NN
      to the study of the hippocampal segmentation (Alzheimer’s
      disease). (WP2)
   – Attività di Testbed e Release&SQA (WP3)
   – Exploiting Bare metal resources (WP4) sia per GPGPU che Low-
      latency network
   – Coordinare l’attività di Hybrid Cloud (WP5)
   – Implementazione di soluzioni di “DEEP as a Service” (WP6)
   – Liaison con i servizi di INDIGO e con i servizi sviluppati in XDC e
      in produzione in EOSC-Hub
                                                                      45
Sommario

   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-HUB (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       46
The eXtreme DataCloud (XDC)

Titolo: The eXtreme DataCloud (XDC)
    • Call: EINFRA-21-2017, topic b: Research and Innovation
      Actions for e-Infrastructure prototypes, subtopic 2:
      Computing e-infrastructure with extreme large datasets
    • Budget totale richiesto: 3.1 M€
    • Coordinatore: INFN (PI: Daniele Cesini)
    • Partecipanti: 8
    • Durata: 27 mesi
    • Budget per INFN: 583 K€, al primo posto
    • Sezioni INFN coinvolte: Bari, CNAF, Padova

Prossime slide: grazie a D. Cesini
                                                               47
The XDC Project
•    develop scalable technologies for federating storage resources and managing
     data in highly distributed computing environments
•    Will be based on existing tools (TRL8+) that will be enriched by new
     functionalities and plugins already available as prototypes (TRL6)
•    Target platforms: current and next generation e-Infrastructures deployed in
     Europe (EOSC, EGI, WLCG and other EINFRA-12-funded infrastructures)

    ID   Partner       Country   Represented Community                   Tools and system that will be developed

    1    INFN (Lead)   IT        HEP/WLCG                               INDIGO-Orchestrator, Smart caching mechanisms,
                                                                        Access pattern analyzer
    2    DESY          DE        Astroparticle Physics, Research with    dCache, Orchestrator, Smart Caching mechanisms
                                 Photons

    3    CERN          CH        HEP/WLCG                                EOS,DYNAFED, FTS, Smart Caching mechanisms

    4    Cyfronet      PL                                                ONEDATA

    5    ECRIN         [ERIC]    Medical data

    6    CSIC          ES        Lifewatch

    7    CNRS          FR        Astro [CTA and LSST]

    8    EGI.eu        NL        EGI communities

                                                                                                                          48
XDC technical topics (1/2)
• Intelligent & Automated Dataset Distribution
   1. Orchestration to realize a policy-driven data
       management
   2. Data distribution policies based on Quality of Service (i.e.
       disks vs tape vs SSD) at infrastructure level (cross-sites)
• Data pre-processing during ingestion
   1. Automatically run user defined applications and
      workflows when data are uploaded i.e. for Skimming,
      indexing, metadata extraction, consistency checks
• Data management based on access partners
   1. Move to ‘glacier-like’ storage unused data, move to fast
      storage “hot” data
   2. access predictions to improve data availability
                                                                49
XDC technical topics (2/2)
• Smart caching
   1. Extension of a site to remote location
   2. Federated and geographically distributed cache i.e.
      Federation-wide caches
•   Advanced metadata management with no pre-defined
    schema
     1. Within ONEDATA
•   Encryption Services and Secure Storage
     1. Sensitive data management and key storage within
          ONEDATA
•   Multi-region support in ONEDATA to federate storage system
    at the Exascale

                                                             50
XDC Project Structure

                                         INFN involved in WP4 to develop:
   INFN involved in all WPs:
                                             The PaaS-Orchestrator
       CNAF, BA, PD
                                                 policy (QoS) driven data
   INFN leads WP1 and WP3                        management
                                                 Pre-processing workflows
   INFN will represent WLCG in WP2          Smart caching mechanisms
                                             Access Pattern Analyzer
                                                                             51
Sommario

   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-HUB (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       52
ICARUS

Titolo: Interoperable eCosystem of historicAl souRces for
eUropean hiStory (ICARUS)
    • Call: INFRAIA-02-2017 (seconda fase)
    • Budget totale richiesto: 4.9 M€
    • Coordinatore: CNR-OVI
    • Partecipanti: 16
    • Responsabile nazionale: L. Gaido (G. Donvito)
    • Durata: 48 mesi
    • Budget per INFN: 493 K€, al secondo posto, dopo il
      coordinator
    • Sezioni INFN coinvolte: Bari, CNAF, Torino
                                                            53
ICARUS

Goal:
• reduce the fragmentation of the historical sources digital
  ecosystem, bringing together and integrating in a rich and
  interoperable Web of Historical Data the existing research
  datasets, allowing scholars and researchers to exploit new and
  powerful technologies as an integral component of the
  historical research methodology

The integration activity will enable seamless access of researchers
to historical datasets, and provide new Web-based services and
tools as well as common interfaces to data repositories.

                                                                 54
ICARUS: ruolo INFN

• Contribuire alla definizione dell’architettura del progetto
• Supportare l’integrazione di alcune componenti di INDIGO
  (IAM, PaaS, Onedata) con il framework per la data analytics
  (D4Science) fornito da CNR-ISTI
• Fornire alcune risorse per i servizi del progetto 
  finanziamento tramite Virtual Access
• Agire da liaison con la EOSC, grazie alla partecipazione al
  progetto EOSC-hub

                                                                55
Sommario

   Il punto di partenza
   EOSCpilot (INFRADEV-04)
   EOSC-HUB (EINFRA-12)
   DEEP-HybridDataCloud (EINFRA-21)
   XDC (EINFRA-21)
   ICARUS (INFRAIA-02)
   SCALE-UP Open DataCloud (ICT-16)

                                       56
SCALE-UP Open DataCloud

Titolo: titolo provvisorio INDIGO-DataExtreme
     • Call: ICT-16-2017, topic: Big data PPP: research addressing
       main technology challenges of the data economy
     • Proposal in preparazione, scadenza call: 25.4 2017
     • Budget indicativo: 5 M€
     • Coordinatore: ATOS
Coinvolgimento di diverse ditte (Bull, Ontotext, EBIT S.r.l. e
probabilmente anche una ditta operante nel campo dei satelliti)
Scopo: indirizzare soluzioni software innovative legate a big data
partendo da INDIGO-DataCloud e dal proposal DEEP-
HybridDataCloud.
Il consorzio comprende inoltre CSIC, INFN, LIP, UPV, ATOS, PSNC,
IISAS (Slovak Academy of Science).                                57
Conclusioni/1
• Le competenze e l’esperienza maturate dall’INFN nei progetti
  Grid fino al FP7, e la posizione di rilevanza a livello
  internazionale sono state mantenute nella prima fase del
  programma H2020 (WP 2014-2015), nonostante il salto
  tecnologico con il passaggio al Cloud Computing. Ne sono
  prova la leadership di INDIGO-DataCloud e il ruolo rilevante in
  EGI_Engage e West-life.
• Nel WP 2016-2017 la rilevanza a livello europeo è confermata
  dal ruolo dell’INFN nei vari proposal presentati  importante
  la capitalizzazione di quanto fatto nei progetti precedenti
• è importante anche a livello nazionale, ma serve una strategia
  comune con gli altri enti di ricerca per fare massa critica
• Per l’INFN i finanziamenti dai progetti europei sono molto
  importanti per ovvie ragioni, ma questo non basta….
                                                               58
Conclusioni/2
• Lo ‘scollamento’ tra quanto fatto nei progetti ICT, in particolare
  quelli finanziati dalla EC, e le attività ‘core’ dell’INFN è stato
  avvertito in modo forte nei progetti Grid (da EGI_Inspire in poi)
• Il riavvicinamento è cominciato già in INDIGO-DataCloud e si è
  rafforzato in EOSC-hub (ad es. Con il TS DODAS)

• Qui le sezioni INFN non direttamente coinvolte nei progetti
  europei possono avere un ruolo importante:
   – negli esperimenti INFN
   – nei resource/data center INFN. E nella Corporate Cloud?
 Ma questo deve essere supportato da una azione decisa di
  dissemination (compreso training) ed exploitation
Questa dovrebbe essere parte della strategia dell’INFN.
                                                                  59
Grazie per l’attenzione!

Per chi è interessato:
dal 9 al 12 Maggio a Catania ci sarà il Summit 2017 di INDIGO
organizzato congiuntamente con la conferenza EGI:

https://indico.egi.eu/indico/event/3249

                                                                60
You can also read