Joint Center for Satellite Data Assimilation Operating Plan 2021
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Joint Center for Satellite Data Assimilation
Operating Plan 2021
DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Executive Summary
This document describes the Operating Plan for the Joint Center for Satellite Data Assimilation
(JCSDA) in 2021. It is intended as a working document to collectively define overlap of interest
and anticipated collaboration across partners and highlight scientific topics where the JCSDA is
expected to have maximum impact. The first part of the document contains a detailed description
of tasks for 2021. The second part of the document presents a high-level description of the five-
year goals. The entire document is updated on a yearly basis.
The JCSDA activities are organized into Projects dictated by common needs across multiple
JCSDA partners, including Disaster-Related Appropriation Supplemental (DRAS) work. For each
Project, a limited set of Tasks are identified, with the intent to focus collaborations from core
resources and in-kind contributions toward specific tangible deliverables. Projects are organized
around three scientific and technical themes: a) Algorithms and Infrastructure, b) Observations,
and c) Applications.
The JEDI project will focus on delivering scientific and computational improvement for the
background error covariance matrix, optimization of advanced solvers (e.g. 4DVar, and block
methods), and ensemble update. Coupled data assimilation will also be an active topic of
research.
The organization of the CRTM project is changing from previous years to reflect the modernized
workflow, applications focus, and refined management practices within the JCSDA. This project
is aiming for consistency with the structure of other projects to enable more seamless and
consistent evaluation of progress and contributions toward the deliverables identified herein.
Activities in the OBS project include development, implementation and testing of forward
operators, bias correction, quality control and error characterizations (Task OBS1). The work will
rely on a set of shared and customizable diagnostic tools (Task OBS2), and an enhanced IODA-
UFO framework (Task OBS3).
The driving principle for the SOCA project will be the staged refactoring of current infrastructure
for the marine components of the UFS, GEOS, and the GFDL MOM6-SIS2 coupled ocean sea
ice model, while keeping the current capabilities required by the ongoing scientific development,
such as weakly coupled DA system for the UFS, re-analysis development at EMC and GMAO,
and regional ocean DA for the initialization of the future NOAA hurricane forecast system.
The focus of land activities at the JCSDA over the AOP21 performance period will be on the
continued development of DA capabilities for the land surface model (LSM) components of
NOAA’s: (i) Unified Forecast System (UFS); (ii) offline UFS CCPP Land Data Assimilation
System (UCLDAS) and; (iii) National Water Model (NWM).
JCSDA Operating Plan 2021_rev 1_sanitized Page 2 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
The recent JEDI-FV3 public release supports UFS and GEOS nascent applications. The ongoing
development of a generic Experiment and Workflow Orchestration Kit (EWOK) and of the
Research Repository for Data and Diagnostics (R2D2) will be expanded to support cycling DA
applications. A suite of generic diagnostic capabilities will be developed, relying on R2D2 for
access to all data. These components provide Infrastructure-as-Code (IaC) environment to run
experiments and share results for a variety of Joint Testbed Applications that will be initiated at
various stages, and configured based on specific agency requirements.
High-level milestones for AOP21 are presented in the table below. Additionally, the attached
Project Plan (see Appendix A) provides more granular information about project tasks,
deliverables, timelines, resources, and responsibilities.
AOP21 Q1 AOP21 Q2 AOP21 Q3 AOP21 Q4
Task High-level Milestone Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
JCSDA Director's
Office
JCSDA Events Technical Review and Science Workshop x
JCSDA Symposium at AMS x
Executive Retreat x
Algorithms &
Infrastructure
Infrastructure Multi-tier automated testing harness x
Optimized performance for IODA observation access x
Application-level performance improvements x
Model interfaces Unified variable transform for B matrix and H operator x
Optimized TLM/ADJ and 4DVar evaluation x
Optimized interpolations x
Algorithms Observation error correlation x
Background error covariance matrix improvements (time
interpolation, spectral) x
Coupled DA capabilities x
Experiments Support full-scale JEDI applications x
Near-real-time observation ingest x
Experiment configuration management x
Observations
CRTM CRTM v3.0 beta release x
Coefficient package release x
JCSDA Operating Plan 2021_rev 1_sanitized Page 3 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
NetCDF lookup tables, NLTE and BRDFs with polarization x
UFO Operators for constituents, TPW x
Operators for ZTD, Sentinel-6, ABI, AHI, MODIS AOD x
Operators for GIIRS, scatterometer winds, and improved QC
for conventional x
Diagnostics Near-real-time observation monitoring x
FSOI for GEOS and GFS x
Impact assessment of commercial data, GEMS2, and OPIR x
Infrastructure IODA naming conventions x
IODA backends to BUFR and ODB x
Applications
Marine Marine UFO release x
Improved interfaces for MOM6, CICE6, and regional ROMS x
Wave and sea-ice initialization x
Land Demonstrate use of snow observations x
Demonstrate use of soil moisture observations x
Near-real-time CONUS SWE analysis x
3D-Var: Static and/or ensemble B, background from oper x
JEDI-SKYLAB
Own background: Run short forecast from oper analysis x
4D-Var: Ensemble B from operations, bckg from short fcst x
Long forecast: Initialize fcst from DA and compute fcst scores x
FSOI: Long fcst and FSOI obs impact assessment x
Cycling Hybrid 4DVar: Deterministic only, get flow dep. B
x
from operational ensemble
Cycling Hybrid 4D-Var + EDA: deterministic & ensemble fcsts x
Weakly coupled variational DA: 4D-Var atmosphere + 3D-Var
x
SOCA (coupled UFS fcst)
Other Apps JEDI-SIMOBS application release x
JEDI-EDU application release x
JCSDA Operating Plan 2021_rev 1_sanitized Page 4 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
CONTENT REMOVED
JCSDA Operating Plan 2021_rev 1_sanitized Page 5 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
CONTENT REMOVED
JCSDA Operating Plan 2021_rev 1_sanitized Page 6 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Projects and Tasks
Project DOF: Director’s Office (Executive Officer: Phil Gibbs)
● Task DOF1: JCSDA Operations
● Task DOF2: Events, Community Engagement, Training and EducationalOutreach
ALGORITHMS and INFRASTRUCTURE (Senior Lead: Yannick Trémolet)
Project JEDI: Joint Effort for Data assimilation Integration (Lead: Yannick Trémolet)
● Task JEDI1: Software Infrastructure
● Task JEDI2: Model Interfaces
● Task JEDI3: Algorithms
● Task JEDI4: Infrastructure for JEDI Experiments
OBSERVATIONS (Senior Lead: Dick Dee)
Project CRTM: Community Radiative Transfer Model (Lead: Ben Johnson)
● Task CRTM1: Software Management andWorkflow
● Task CRTM2: Model and Application Development
● Task CRTM3: Science Development and Application Outcomes
Project OBS: Improved use of Observations (Lead: Dick Dee)
● Task OBS1: UFO implementations
● Task OBS2: Diagnostic tools and workflows
● Task OBS3: Software Infrastructure
APPLICATIONS (Acting lead - Tom Auligné)
Project SOCA: Sea-ice, Ocean, Coupled Assimilation (Lead: Guillaume Vernieres)
● Task SOCA1: Marine model interfaces
● Task SOCA2: Marine Applications
Project LAND: Land and snow data assimilation (Lead: Andy Fox)
● Task LAND1: Land Applications
● Task LAND2: Land Observations
Project CHEM: Atmospheric constituents data assimilation (Lead: TBD)
● Task CHEM1: Aerosol and Reactive Trace Gases
Project APP: Joint Testbed Applications (Lead: TBD)
● Task APP1: Initiate Joint Testbed Applications
JCSDA Operating Plan 2021_rev 1_sanitized Page 7 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Part I: AOP 2021
JCSDA Annual Operating Plan April 1, 2021 -
March 31, 2022
JCSDA Operating Plan 2021_rev 1_sanitized Page 8 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Task DOF1: JCSDA Operations
The JCSDA’s mission will be fulfilled by providing scientific leadership and development
opportunities in order to fully exploit the assimilation of satellite data for operational and
research environmental analyses and predictive models and to track the progress of the JCSDA
scientific projects. The Director and the Director’s Office Team will facilitate internal and external
coordination through project management best practice to provide day-to-day leadership,
management, and administration of the JCSDA to ensure on-going interagency partner activities
are directed toward the mission goals and related annual deliverables of the JCSDA.
DOF1.1 - Scope Management
● Collect and derive the requirements for JCSDA AOP21 projects, 1 year and 5 year plans.
● Work with sponsor agencies and partners to define the scope of AOP21 projects, 1 year
and 5 year plans.
● Work with sponsor agencies and partners to create the Work Breakdown Sheets for all
projects; and obtain approval of the JCSDA AOP21.
● Perform configuration and change control of the approved AOP21 project scope.
DOF1.2 - Schedule Management
● Plan the schedule management of AOP21 project deliverables; define activities,
resources, sequence, and duration of activities.
● Develop the schedule as a high level Gantt Chart, and monitor and control at lower levels
using the ZenHub project management tool.
● Perform configuration and change control of the approved AOP21 project schedule.
DOF1.3 - Agile Project Management through ZenHub
● Plan and manage project boards, repositories and workspaces in ZenHub.
● Manage the Agile methodology process flow.
● Hold monthly check-ins with project leads as a measure of team engagement,
milestones progress, change control (requests), risks and/orissues.
● Provide guidance and assistance for code sprints within each team, daily stand-up and
retroactive review process.
DOF1.4 - Release Management
● Draft tailored process for JCSDA software applications release to include the following
standard processes:
JCSDA Operating Plan 2021_rev 1_sanitized Page 9 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
○ Functional product process; Release packaging process; Documentation process;
Development process; Change control process; Customer testing process;
Customer notification process; Training process; Deploymentprocess
DOF1.5 - Cost Management
● Plan JCSDA AOP21 budget related to work, based on existing awards and funding.
● Work with UCAR finance office to draft new budget allocations forAOP21.
● Create and manage JCSDA account keys for existing and newfunds.
DOF1.6 - Risk Management
● Plan risk management process to identify risk, perform qualitative and quantitative
analysis, plan risk response/mitigation.
● Application of the risk management process through monthly risk meetings with all risk
owners and approved change requests.
DOF1.7 - Project Reporting
● Plan reporting process and cadence for AOP21 projects including:
○ Quarterly reviews; DRAS quarterly and monthly reports; additional project monthly
reporting requirements.
DOF1.8 - General Administration/Office Management
● Provide general administrative support to the JCSDA team, both local and distributed
● Coordinate with UCP Business Shared services to provide domestic/international travel
support and help with general administrative overflow
● Provide general support to the director in the form of scheduling meetings, coordinating
travel.
● During COVID-19, coordinate UCAR safety training and buildingaccess
● Attendance of UCP administration meetings and meetings for administrative applications
(e.g., Workday employee management system) to keep current with updates, rollouts,
and processes.
DOF1.9 - Human Resource Management
● Coordinate across UCAR HR (Talent Acquisition) and UCP Business Shared Services to
identify and adhere to the official hiring process for the JCSDA.
● Work with UCAR HR Composition and UCP Business Shared Services to complete tasks
for JCSDA promotions, reclassifications, and visitor/contractorpositions.
● Complete the hiring process for all new JCSDA employees and visitors:
○ Position description iteration, selection of candidates, interviews, offerletters.
● Manage the onboarding process for all new JCSDA employees andvisitors:
○ Google account set up, order and ship computers/peripherals, add to shared
documents/calendars/aliases, assist with timecard entry, assist withvisas.
JCSDA Operating Plan 2021_rev 1_sanitized Page 10 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
● Complete the termination process for all JCSDA employees andvisitors:
○ Gather all equipment/badges, inform payroll, initiate termination in Workday,
coordinate with IT to shut down Google accounts, remove from all
aliases/calendars/documents.
● Maintain up-to-date records of JCSDA position descriptions, employee visa statuses,
supervisor training statuses, federal employee badgestatuses.
● Coordinate the onboarding with federal agency sponsors for off-site employees and
visitors:
○ Federal security procedures, badging, provision of workspace, equipment, IT
infrastructure.
DOF1.10 - Procurement
● Perform and oversee project procurements such as HPC allocations, materials and
supplies for the JCSDA team, various purchased services (e.g., software licenses).
● Manage the invoicing and approvals process for current purchase orders for subawards,
contract workers, and purchased services (e.g., JCSDA Quarterly Newsletter).
● Purchase and track all sensitive property within the JCSDA such as laptops, desktops
while adhering to UCAR's department property administrator policies:
○ Conduct annual property audits and attend required training, and meetings on a
regular basis.
● Purchase items for the JCSDA using a purchasing card:
○ Reconcile P-card charges monthly and attend annual training sessions.
● Create payment requests for employees and visitors for conference registration, abstract
fees, reimbursements.
DOF1.11 - Communication and Stakeholder Management
As the JCSDA continues to expand, there is increasing interest in the latest project
accomplishments. A strategic communication plan, in addition to effective channels of
communication, will ensure that the cutting-edge work coming from the JCSDA is communicated
in a timely and understandable manner to a variety of audiences.
● Schedule and conduct various stakeholder meetings:
○ Weekly review meetings with project leads to allow for open communication
between teams and management.
○ Weekly administrative check-in meetings with the Chief Administrative Officer to
foster communication between hubs.
○ Quarterly Management Oversight Board and Executive Team meetings to discuss
high-level JCSDA matters such as funding, project progress.
○ JCSDA All-Hands meetings with the core and in-kind JCSDA staff to
communicate AOP allocations and project updates.
● Coordinate JCSDA science cookies (weekly Monday tag-up) to strengthen team building
and share science:
JCSDA Operating Plan 2021_rev 1_sanitized Page 11 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
○Create and maintain the schedule, send weekly reminders of upcoming
presentations.
● Manage JCSDA Quarterly Newsletter production for the annual operating year:
○ Keep an updated production schedule, meet regularly with the contractor and
editor, provide editing input, post final product on the website.
● Maintain the JCSDA website:
○ Generate blog posts, update the event calendar, update general information and
job postings
● Create, manage and maintain all JCSDA email aliases (stakeholderregister).
● Coordinate a monthly contribution to the UCP Newsletter, frequent JCSDA news blasts,
and regular content for new communication partner channels (e.g., UFS community
updates).
Task DOF2: Events, Community Engagement, Training
and Educational Outreach
DOF2.1 - Academic, Scientific, and Community Engagement Events
The JCSDA is a center of excellence that provides ample opportunities for stakeholders, the
public, and interested communities (academic, scientific, technical) to connect with project work.
Several events are held throughout the operating year to showcase JCSDA work, demonstrate
project progress, and promote networking across communities.
Task: plan, organize, and run several events to foster community engagement:
● 18th JCSDA Technical Review Meeting and Science Workshop (virtual, June 2021)
● 10th AMS Symposium on the JCSDA at the AMS Annual Meeting (January2022)
● Annual JCSDA Executive Retreat (February2022)
● JCSDA Summer Science Colloquium (planning starts this annual operating plan, Summer
2022)
DOF2.2 Training Opportunities
One of the main goals of training and educational outreach opportunities produced by the JCSDA
is to show the community at large how to effectively utilize our open source tools to advance
atmospheric and data assimilation science. To maintain excellence and encourage community
development and engagement, the JCSDA offers several training opportunities to in- kind
contributors, the general public, and other interested parties within academic, private sector, and
science communities. In-person or virtual training events for JEDI and CRTM will be
supplemented by publicly accessible online tutorials. Internal training will be provided to JCSDA
core and in-kind staff to support timely software releases. During this operating year, work will
begin on a JEDI-EDU web application.
JCSDA Operating Plan 2021_rev 1_sanitized Page 12 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Task: plan, organize, and run various training opportunities to promote collaboration and
community development:
● JEDI Academies (virtual or in-person, two per operating year)
● JCSDA Executive Academy (JEDI Academy designed for seniormanagement)
● Mini JEDI Academies and Short Courses (AMS)
● Project training sessions (e.g., CRTMworkshop)
● Training sessions and material for agile methodology, release management, and CI/CD
Task JEDI1: Software Infrastructure
Task Description
The most fundamental responsibility of the software infrastructure team is to provide the tools
necessary to build JEDI applications and their dependencies (the jedi stack) from source.
Though we do not expect the JEDI dependencies to change substantially moving forward, some
maintenance is needed to ensure that software versions remain up to date and mutually
consistent. Thus, we will continue to maintain up-to-date scripts to build JEDI and its
dependencies on a range of systems. This includes explicitly maintaining JEDI environment
modules on Hera, Orion, Discover, Cheyenne, S4, and Gaffney (JEDI 1.1) and maintaining the
necessary infrastructure to build and run JEDI experiments and applications on the commercial
cloud (JEDI 1.2). We will also continue to use the same set of build scripts to produce and
distribute a series of software containers (JEDI 1.3) to accompany each JEDI release, spanning
several container platforms (Docker, Singularity, Charliecloud), several compiler options
(gnu/openmpi, clang/mpich, intel/impi), and several use cases (continuous integration testing,
development, tutorials, and multi-node HPC applications).
In 2021 the JEDI1 team will assume primary responsibility for the IODA subsystem, including
optimization (JEDI 1.5) and extension to support more observation types and to support
development in ioda-converters and ufo (JEDI 1.4). Task 1.5 encompasses parallel I/O, layout of
data in files, efficient memory management (e.g. exploiting HDF5 capabilities such as
“chunking”) and optimal distribution of observations in core, weighing the trade-offs between
minimal inter-processor communication and maximal load balancing. The JEDI 1.5 deliverable is
to achieve performance comparable to operational systems such as GDAS where meaningful
comparisons can be made. Meanwhile, the deliverable for JEDI 1.4 is to provide the IODA
infrastructure needed to support all observation types and file formats (including data layouts
optimized for diagnostics) that are needed for application releases.
The optimization of IODA on the repository level will be accompanied by other specific repository
optimization tasks in other JEDI projects. However, it is also essential to monitor and improve the
overall system-level performance of JEDI applications and DA workflows (JEDI 1.6). The JEDI1
team will define a consistent set of profiling tools and an organizational framework to assess and
progressively improve such system-level performance across HPC systems for each JEDI
application that is scheduled for release. Included in this system-level
JCSDA Operating Plan 2021_rev 1_sanitized Page 13 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
optimization is a move to single-precision in selected code components to improve computation
and memory efficiency. An optimization code sprint will be scheduled to work with the application
teams to implement this optimization framework and to identify and address bottlenecks. Since
performance bottlenecks generally vary with application configurations (e.g. model resolution,
observation count/distribution, etc.), this must be done within the context of a systematic
hierarchy of progressively more sophisticated application workflows (JEDI1.7).
The goal of the JEDI 1.7 task is to provide generic standards and tools for hierarchical JEDI
application suites. This includes a multi-tier Continuous Integration (CI) testing framework in
which low-resolution (Tier 1) tests are run for every GitHub pull request (already implemented),
and more computationally intensive tests (Tiers 2 and 3), involving higher-resolution models,
more observations, ands more comprehensive workflows, are run nightly or weekly. This
hierarchy of coupled applications and resources (e.g. data and configuration files) will also be
leveraged for development and profiling purposes, allowing developers to progressively enhance,
test, optimize, and deploy a series of applications of increasing sophistication and scope. JEDI
users will also be able to benefit from this hierarchy, selecting application configurations that best
meet their needs for research, operations, or teaching.
Another important aspect of CI that leverages the added value of the Joint Center is the concept
of testing pipelines. When a change is made to a base repository such as OOPS or IODA, an
automated pipeline will assess its impact on other repositories by building their applications and
running their tests. A deliverable for AOP 2021 (JEDI 1.8) is to implement a comprehensive CI
pipeline to support each application release.
All JEDI team members will share responsibility for code reviews and user support (JEDI 1.9).
Multi-level user support will be administered through Zenhub project boards, public forums
(https://forums.jcsda.org), and aggregated insights (FAQ, wikis and/or knowledge bases), with
the JEDI1 team taking the lead on implementation. JEDI documentation and training will also
now fall under the responsibility of JEDI 1 (JEDI 1.10). We will continue to organize two JEDI
Academies per year and we will provide a series of online tutorials in support of each application
release. And, JEDI1 will help ensure that the online JEDI user manual is up to date with the latest
releases.
The JEDI1 team will also play a key role in the effort to release a JEDI-EDU release (APP1),
dedicated to supporting graduate-level training in data assimilation at colleges and universities.
Task JEDI2: Model Interfaces
Task Description
Rapid progress has been made in interfacing the generic components of the JEDI system with
several sophisticated operational and research atmospheric models (GFS, GEOS, FV3-SAR,
NEPTUNE, MPAS, LFRic and UM). This work involves writing the software interfaces
JCSDA Operating Plan 2021_rev 1_sanitized Page 14 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
specifically, for the model grid, fields and variables and constructing all the configuration for
testing and building various applications. All major data assimilation algorithms have been
developed across the various models and in many cases deployed in cycling workflows. In this
task the effort to develop these interfaces is continued, with a primary objective of preparing the
software for operational use, both in terms of accuracy and efficiency. A secondary objective of
this task is to consolidate the efforts across the interfaces and models to bring a more unified
approach to interface development. There is considerable duplication of code across model
interfaces and this should be reduced where possible.
Much of the focus of the work in JEDI2 will be on the interface to FV3-based models GFS and
GEOS, handled in the FV3-JEDI interface. Work will include providing more complicated data
structures for the observation operators, including field of view and slant path interpolations.
Providing these will also require infrastructure changes to generalize the Locations and GeoVaLs
interface classes in UFO. Ensuring all interpolation routines are efficient for FV3- based models
will require improvements to the interpolation routines so that knowledge of the grid is utilized.
Work will be conducted to include the cubed-sphere grid in the Atlas library, making numerous
efficient interpolation and grid tools available for use in FV3-JEDI as well as the LFRic interface.
The VAriable DErivation Repository (VADER) will be introduced as a way of sharing variable
transforms between model interfaces. This will prevent repetition across interfaces and simplify
the grouping of variable transforms. An additional issue that creates complexity is the naming of
variables, which can differ between the model interfaces and the UFO. A new unified library of
variable names will be introduced so that all interfaces acquire the same naming conventions for
field metadata.
Much of the generic infrastructure for generating the hybrid background error covariance model
has been developed. The focus this year will be on building and tuning the components of the
operators for all the models. This will include refining the horizontal and vertical balance
operators and tuning the coefficients in the covariance and localization models to produce
realistic increments. Work will be conducted to compare the cubed sphere background error
models with existing models.
Operational requirements will include the need for ensemble data assimilation using the LETKF.
In addition to the work on the background error covariance, work will be conducted to
scientifically validate ensemble data assimilation using the LETKF and GETKF methods.
A sophisticated adjoint model for FV3 as well as GEOS physics packages is available through
the FV3-JEDI interface. Work will be conducted to improve the efficiency of this system so an
operationally viable 4DVar system can be tested and implemented. A generic software
infrastructure will be put in place to generate the adjoint of the physics of any model using an
ensemble and the LETLM methodology. A detailed comparison of 4DVar and 4DEnVar will be
conducted.
JCSDA Operating Plan 2021_rev 1_sanitized Page 15 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Model interfaces for MPAS, NEPTUNE, UM, LFRic and a shallow water model will continue to be
developed throughout this year. Work will include keeping the interfaces up-to-date with changes
to the generic parts of the JEDI code and utilizing developments such as the improved data
structures in the Locations class.
Task JEDI3: Algorithms
Task Description
The main responsibility of the Algorithms team is developing and supporting generic data
assimilation algorithms that can be used with different model interfaces and different
observations. Multiple algorithms are already available in JEDI system: a variety of variational
algorithms (3DVar, 4DVar, weak-constraint 4DVar; with possibility of using static, ensemble, or
hybrid (with any number of components) background error covariances); EDA (ensemble of data
assimilations), LETKF (local ensemble transform Kalman filter) and gain form of LETKF. This
year work will be focused on improving and supporting these existing data assimilation
algorithms, and on adding generic coupled data assimilation capabilities.
Variational algorithm improvements will focus on adding new features (e.g. middle loop),
improving performance and ease of use of variable changes, and simplifying the code (JEDI3.1
and JEDI3.2). Experiments using EDA with different models will determine further development
for EDA (JEDI3.4 and JEDI3.5).
For the background error covariances, JEDI3 task will focus on the spatial and time covariances
this year (multivariate covariances and balance operators will be part of JEDI2 task).
Generic time localization will be added (JEDI3.8). For spatial covariances, work is planned for
computational optimization of the existing BUMP covariances (both for parameter estimation,
and for the application of the covariances) (JEDI3.10). The effort on implementation of other
spatial covariance operators will continue, and the goal this year is to have the implementations
in the saber repository, so they could be used by different models (JEDI3.11-JEDI3.12).
In 2021 the team will continue development of the LETKF-type ensemble Kalman filters,
including computational improvements and adding new features like using linearized observation
operators for the ensemble perturbations (JEDI3.7).
Time interpolation in observation operators, implemented for um-jedi in AOP 2020, will be
generalized for use with any model in JEDI3.9 task.
Significant effort will be dedicated to advancing coupled data assimilation generic capabilities in
JEDI (JEDI3.6), including improving on coupled H(x) application developed in the previous year,
adding coupled interface classes, and coupled variational assimilation. JEDI3.6 will include
testing these developments with toy models. JEDI3.6 will also include adding an option of
estimation of coupled background error covariances in BUMP.
JCSDA Operating Plan 2021_rev 1_sanitized Page 16 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
JEDI3 will also include generic code improvements to allow for use of VarQC (JEDI3.2),
variational bias correction (changes in preconditioning, JEDI3.3), and work on cross-channel
correlated observation errors (JEDI3.13). JEDI3.14 will interface JEDI and TensorFlow
parameters to allow the insertion of trainable models.
Core team members will dedicate part of their time to general maintenance, bug fixes and
preparation for releases in the oops and saber repositories (JEDI3.15).
Task JEDI4: Infrastructure for JEDI Experiments
Task Description
Over the past few years the JEDI data assimilation system has matured to the point that
operational grade experiments should be performed for validation of the system. Whereas
developments in past years were based on very limited datasets, this next step requires access
to full datasets, and to operational-like computing resources. Because of the collaborative nature
of JCSDA the infrastructure to access such resources has to be very flexible and generic with
respect to the model being run, the computing platform available from each partner, and the
environment (workflow engine) available on those platforms. In the past year, progress has been
made in this direction with the implementation of the initial versions of the Experiments and
Workflows Orchestration Kit (EWOK) and the Research Repository for Data and Diagnostics
(R2D2). In the planning period, these two tools will be further developed to fully support full scale
testing of JEDI and the JEDI applications JCSDA and its partners are initiating.
The first obstacle to portability for cycling DA experimentation is access to data. R2D2 is
providing a uniform access to observation and model data, with an interface that can be used in
generic tasks or even on the command line for retrieving data for diagnostics. The actual storage
and retrieval of the data is handled by R2D2, either from cloud storage or from a local store for
the main HPC systems used for JEDI developments by the JCSDA partners.
The main design idea driving EWOK development is that most suites can be written in an
abstract sense, just like application algorithms in OOPS, with each task being implemented
either specifically for a model, or everywhere possible in a generic manner. EWOK then
generates an actual suite, for a specific model, and a specific workflow engine (ecflow or cylc
initially). Prototypes of R2D2 and EWOK exist for running observation monitoring experiments
“H(x)” in retrospective mode. They will be extended to full DA cycling, including ensemble
components and coupled systems, on dedicated HPC and cloud platforms.
One of the first applications of an EWOK suite will be the near real time ingest of observation
data into R2D2, requiring additional modes to trigger tasks in an operational-like mode, based on
time of day or external events, not just on completion of other tasks in the suite as in
retrospective experiments. This suite will validate EWOK for potential operational use in the
future, and will populate R2D2 for all JCSDA and partner experiments to use. Prototypes will be
JCSDA Operating Plan 2021_rev 1_sanitized Page 17 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
tested with several data sources, such as the NOAA Data Lake that uses cloud storage, and
other data sources provided by the JCSDA partners.
The next step of development will address the storage and management of experiments
configurations so that scientists can very easily copy, modify, run and compare experiments. The
data generated by experiments will be made available through R2D2 for diagnostics (see
OBS2.1).
Finally, tools to monitor computing resources will be developed and integrated into EWOK and
R2D2. Cloud security aspects will be evaluated and appropriate action taken to ensure JCSDA
resources are properly used.
Task CRTM1: Software Management and Workflow
Task Description
The fundamental responsibility of this task is to provide the tools necessary to build, test, release,
distribute, and maintain the CRTM and associated applications and their dependencies, from
source. This is accomplished through the JCSDA established standard practices and procedures
of CI/CD, the use of the JCSDA Github repository and issue tracking, and Zenhub for project
organization and management.
CRTM dependencies are not expected to change substantially moving forward, however some
maintenance is needed to ensure that software versions remain up to date and mutually
consistent; with additional responsibilities for the handling of large lookup tables (netCDF4,
binary). Thus, we will continue to maintain up-to-date scripts to build, test, deploy, and maintain
CRTM / Applications and their dependencies on a range of systems and environments. This
includes explicitly maintaining CRTM build capabilities on Summit, Hera, Orion, Discover,
Cheyenne, S4, AWS, and on Linux/MacOS personal computing environments. Additional
environments may be added over time. There will be coordination with other JCSDA projects
such as OBS and JEDI, along with in-kind contributions from partners that will form the basis for
our testing and evaluation of newly developed software.
We will also continue to use the same sets of build scripts, developed in CRTM v2.4.0, to
accompany each CRTM release covering several compiler options (gnu, clang, intel, PGI
(limited), IBM/cray (limited)), and several use cases such as continuous integration testing,
development, tutorials, and multi-core HPC applications. This includes explicit support for both
cmake and ECTools-based build environments. The legacy autotools build environment will be
deprecated for CRTM v3.0.
In 2021 the CRTM team will assume primary responsibility for software and workflow
optimization and necessary extensions to support a broader range of satellite observations
through the coefficient generation package and coefficients generated by the package.
JCSDA Operating Plan 2021_rev 1_sanitized Page 18 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
The CRTM team will also work with the OBS team to utilize data from the NOAA Data Lake
project into JCSDA DA workflows, which will form the initial basis for the JEDI-SIMOBS
application.
In coordination with the JEDI team, the CRTM team will also define a consistent set of profiling
tools and an organizational framework to assess and progressively improve such system-level
performance across HPC systems for each CRTM-specific application that is scheduled for
release.
In support of JEDI / UFO requirements, the CRTM will be tested and evaluated in both double
and single precision modes for speed and accuracy, these findings will be reported to all
stakeholders. We plan to participate in a planned JEDI code sprint related to this issue to rapidly
facilitate this capability.
The JCSDA CRTM team is responsible for all official software version releases. The goal of task
CRTM1.4 is to lay the groundwork for a generic hierarchical development and release system for
CRTM and applications. This includes a multi-tier Continuous Integration (CI) testing framework
(task CRTM1.5) in which low-resolution (Tier 1) tests are run for every GitHub pull request, and
more computationally intensive tests (Tiers 2 and 3), involving more comprehensive tests
spanning more sensors and larger profile datasets -- the primary goal being to exercise most of
the key elements of the CRTM code. The specific implementation will be primarily adopted from
best practices in the JEDI team in coordination with key personnel.
This hierarchy of coupled applications and resources (e.g. data and configuration files) will also
be leveraged for development and profiling purposes, allowing developers to progressively
enhance, test, optimize, and deploy a series of applications of increasing sophistication and
scope. CRTM users will ultimately also be able to benefit from this hierarchy, selecting
application configurations that best meet their needs for research, operations, or teaching.
Another important aspect of CI that leverages the added value of the JCSDA is the concept of
"testing pipelines". When a change is made to CRTM, an automated pipeline will assess its
impact on itself and other repositories by building the respective applications and running their
tests.
All CRTM team members (core and in-kind) will share responsibility for code reviews and user
support (CRTM1.6). Multi-level user support will be administered through Zenhub project boards,
public forums (https://forums.jcsda.org), and aggregated insights (FAQ, wikis and/or knowledge
bases), with the responsible CRTM team members taking the lead on developing and
contributing to their respective tasks .
CRTM documentation and training (CRTM1.7) will continue as in previous years, but will make
better use of the aggregation of statistics and information contained in the various JCSDA
repositories through associated github/zenhub issues. We will organize one CRTM training
JCSDA Operating Plan 2021_rev 1_sanitized Page 19 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
activity per year, and continue to provide / update a series of online tutorials in support of each
release and/or application.
The CRTM has also led the way in developing a graduate student test for running CRTM in
stand-alone mode, the initial graduate student tests will be expanded to take advantage of the
new PyCRTM interface to further improve the ease-of-use of the software and applications.
Task CRTM2: Model and Application Development
Task Description
This task primarily focuses on the sub-tasks contributing to the code-intensive development of
the CRTM software, tools, and associated applications. Whereas CRTM1 focuses on
management and support, this task focuses on the actual coding, modification, and creation of
core elements of the CRTM software projects.
Four primary code-intensive development efforts will occur throughout AOP2021.
First, and foremost is the finalization of the CRTM v3.0.0 release, and any subsequent minor
updates involving bug-fixes.
Second is the CRTM Coefficient Generation Package, which is used to compute the
transmittance coefficients for the multitude of sensors supported by the CRTM. This release is
expected early in AOP2021 (task/deliverable CRTM2.7).
Third is the JEDI-SIMOBS application, which will be used as a standalone interface to compute
CRTM v2.x and v3.0 forward operator and Jacobian directly from atmospheric profiles, without
the need for any model background.
A fourth application is planned as an incubator for AI-based improvements to the CRTM called
CRTM-AI (task CRTM2.8). The goal of CRTM-AI is to facilitate the transition of computationally
intensive elements to an AI/ML approach, with the goal of accelerating computations with a
minimal impact on accuracy for both the forward operator and Jacobian output. This effort will be
coordinated with planned AI/ML efforts within the JCSDA and with key partners to minimize
duplication of effort.
JCSDA Operating Plan 2021_rev 1_sanitized Page 20 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Task CRTM3: Science Development and Application
Outcomes
Task Description
As in prior AOP years, CRTM3 is focused on science development. AOP21 adds a new
component focused on the outcomes of applications.
The capability of the CRTM to accurately reproduce satellite observations across all-weather
situations depends directly on the accuracy of the underlying models and assumptions. For clear
sky radiances, the CRTM produces highly accurate simulations of satellite sensor observations,
typically within instrument precision for infrared (IR) and microwave (MW) instruments. However,
for cloudy and aerosol impacted scenes, this accuracy decreases substantially due to two
factors: (1) the ability of the calling model to properly represent the physical state of the
atmosphere and surface; and (2) uncertainties in the physical models / parameterizations within
CRTM, and uncertainties associated with the radiative transfer solver itself.
Task CRTM3 aims at continuous improvement of the physical assumptions within the CRTM to
enable more accurate simulation of radiances in all-sky all-sensor conditions. Because CRTM is
a 1-D model, it has limitations related to satellite viewing geometry, such as 3-D effects, and
surface emission/reflection effects at high viewing angles.
For water and ice clouds, Task CRTM3.1 aims at expanding the wavelength and effective radii of
the existing cloud table to cover a broader range of sensors, particularly in the MW region up to
664 GHz. Additionally, by expanding the effective radius, we also are able to more accurately
simulate large hydrometeors, such as observed in heavy precipitation. Within the JCSDA and in
coordination with Penn State University researchers, we have developed a new set of
coefficients designed to align the physical assumptions in CRTM with the cloud microphysical
model assumptions in the calling model. This effort will continue to be expanded to support full
polarization computations, and updated as needed for current and future sensors.
For aerosols, Task CRTM 3.1 also similarly expands the number of species, ranges of effective
radii, and supported wavelengths to more completely cover the range of variability in observed
aerosols.
In CRTM v3.0, polarization information from land, ocean, and ice-covered surfaces is critical to
accurately simulating observed satellite radiances. Tasks CRTM3.4 and CRTM3.8 aim at
improving the lookup tables (LUTs) used within the CRTM across the entire spectral range from
UV to MW, with a specific emphasis on developing accurate surface BRDF LUTs across various
surface types. For visible / near-IR wavelengths, task CRTM3.4 will explore alternative solver
solutions to accelerate the CRTM computations, which are traditionally slower than thermal-only
JCSDA Operating Plan 2021_rev 1_sanitized Page 21 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
simulations. These efforts tie directly into task CRTM3.8, which aims at improving the interface
and utilization of the Community Surface Emissivity Model (CSEM).
Improvements to the Zeeman and NLTE simulation capabilities of the CRTM will continue in
collaboration with in-kind and external collaborators via tasks CRTM3.5 and CRTM3.6,
respectively.
A new component of the CRTM research portfolio is represented by Task CRTM3.9, which
specifically aims at developing a validation framework for evaluating the wide range of sensors
that the model simulates. Specifically, we plan to use existing independent satellite validation
campaign data to refine the CRTM physical assumptions and to evaluate any shortcomings in
the computed quantities.
Tasks CRTM3.10 - CRTM3.14 are the tasks and deliverables associated with applications under
development by the CRTM team. In particular, the SimObs package (using CRTM v3.0), CRTM
v3.0, and CRTM-AI output will be evaluated using the framework developed in task CRTM3.9,
and by using existing tools for profiling the model running under a variety of conditions and
applications. The CRTM-AI components developed in 2021 will be evaluated against CRTM v3.0
for timing and accuracy (Task CRTM3.14).
Task CRTM3.13 represents the ongoing / continuous task of satellite sensor coefficient
generation. As workflow and capabilities for the transmittance coefficient generation package
improves, we expect to be able to routinely produce, evaluate, and deliver new instrument
coefficient files rapidly. Coordination with the OBS team and JCSDA partners will determine the
priority for sensor coefficient files to be created.
Task OBS1: UFO Development
Task Description
The purpose of this Task is to develop, test and make available a comprehensive collection of
capabilities for the Unified Forward Operator (UFO) for all observing systems relevant to Earth
system analysis and prediction. The scope includes all observing instruments that are used in
operations by JCSDA partners; instruments that are currently active and planned to be used (or
considered for use) in operations; legacy instruments that are no longer active but important for
use in climate reanalysis. It may also include future observing systems based on new
technologies that are still under development.
Activities in this task include development, implementation and testing of forward operators, bias
correction, quality control and error characterizations (Task OBS1). The work will rely on an
enhanced IODA-UFO framework (Task OBS3) involving R2D2 for access to all data
(observations and model output) and a set of shared and customizable diagnostic tools (Task
OBS2).
JCSDA Operating Plan 2021_rev 1_sanitized Page 22 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
The use of R2D2 (Repository for Research Data and Diagnostics) will accelerate the introduction
of new observations in JEDI and play a central role in UFO development and testing. It will
greatly simplify access to observations, model backgrounds and other data needed for data
assimilation experiments and generation of diagnostics. R2D2 abstracts data access by using
specifying key/value descriptors rather than paths or filenames, so that users need not be
concerned with physical file locations. R2D2 can store data in the cloud and/or locally.
Developments in JEDI4 will ensure that R2D2 provides reliable access to NRT observation data
and model background fields.
Process and steps for introducing new observations in JEDI and diagnosing their impact in a DA
experiment. An important goal for the OBS project in AOP2021 is to streamline this process and
remove bottlenecks in order to make it scalable and efficient for use in JEDI applications. The main
code repositories to be managed by the OBS team are IODA-converters, UFO, and DIAG. The
team will also contribute to IODA and R2D2 repositories.
UFO development is separated into four subtasks, addressing constituents and aerosols
(OBS1.1), satellite radiances (OBS1.2), data from Global Navigation Satellite Systems (OBS1.3)
and data from conventional instruments and satellite products (OBS1.4).
Development and testing will be accelerated by conducting frequent code sprints designed to
address the current priorities of JCSDA partner agencies, focussing on specific groups of
observing instruments and/or shared science questions. The code sprints will be planned and
managed by the core team to ensure that requirements and end-goals are clearly defined in
advance and shared among all participants. The core team will also ensure that all necessary
data, information and tools are in place at the start of each code sprint, and that collaborative
development and testing follows well-defined protocols and work practices.
OBS1.5 provides support to the JEDI Core and Application Teams, e.g. by supplying tailored
UFO configurations for the development of JEDI NWP Testbed Applications, producing UFO
modules for community JEDI releases and user training.
OBS1.6 ensures that OBS team members routinely and systematically inspect the NRT flow of
observation data using the NRT Observation Monitoring Application (developed in OBS2).
JCSDA Operating Plan 2021_rev 1_sanitized Page 23 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
The primary purpose is to detect issues that can affect the performance of all JEDI
applications, due to issues with data access or quality control. Additionally, routine
monitoring of the various observing instruments and their impact on DA performance will
create opportunities for innovation and improvements in UFO.
Task OBS2: Diagnostics
Task Description
The goal for this task is to develop and make available an open and shared infrastructure for
exploring and diagnosing observations and their impact on data assimilation system
performance. The infrastructure will be implemented in the cloud and rely on data access via
R2D2.
OBS2.1 is dedicated to the development of diagnostic tools based on a prioritized list of
diagnostic requirements. Python-based tools will be delivered in the form of Jupyter
notebooks that can be shared and customized for interactive use (plotting on demand). A
dash boarding solution will be provided for displaying visualizations and diagnostics
generated by data assimilation experiments, including JEDI Testbed Applications. All
diagnostic software and tools will be consolidated in a newly established DIAG repository.
OBS2.2 will build and maintain a new CI/CD web application for NRT observation monitoring
and UFO performance diagnostics, to be implemented on AWS using R2D2 for data access,
EWOK for workflow management and DIAG for generating diagnostics. The application is
intended as a platform for exploring observation data used in Earth system analysis and
prediction. It will serve as a front-end for R2D2, with a continuously refreshed visual display
of the current global observing systems, and providing access to its catalogue of observation
data, to the available UFO configurations, and available diagnostic tools.
Capabilities for impact assessment of observation data are addressed in OBS2.3 and
OBS2.4, including work on developing protocols and tools needed to enable rapid
assessment of (potential) impact of specific (future) observing systems on DA performance in
the NWP context. The goal is to integrate assessment capabilities within the CI/CD web
application to be developed in OBS2.2. Work will continue on implementing and testing of
various FSOI options in data assimilation. In addition, several commercial data studies will be
conducted in OBS2.5.
JCSDA Operating Plan 2021_rev 1_sanitized Page 24 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Task OBS3: Observation infrastructure
Task Description
This task addresses development of infrastructure, coding and software engineering
elements to ensure that all UFO development and work on diagnostics follows best practices,
produces efficient and portable code, and is consistent with the overall JEDI design and
implementation. It includes management of the main code repositories involved in
observation handling (UFO, IODA-converters, DIAG) and providing contributions to the
IODA, R2D2, and UFO code repositories in coordination with the JEDI coreteam.
OBS3.1 concerns upkeep tasks for the UFO, IODA-converters and DIAG repositories,
including pull request management, code documentation and testing protocols, preparation
of code releases, and support in the form of tutorials and JEDI Academy modules. OBS3.2
encompasses work on IODA and IODA-converters to improve data ingest into JEDI. IODA
naming conventions will be developed in coordination with the partner agencies that will
standardize variable names and units where appropriate. The naming conventions will be
implemented and eventually merged with the common variable library for model and
observation interfaces (to be developed in JEDI2). Existing IODA converters will be updated
to reflect any IODA changes, and new converters will be added to ingest additional data
sources. Validation tools will be developed to ensure that data standards are met. This task
will also extend IODA with the features necessary to process both BUFR and ODB file
formats in memory (i.e. without writing intermediate files to disk). IODA and R2D2
development will be coordinated so that observation data can flow seamlessly between the
two codes.
Several infrastructure updates for UFO will be developed in OBS3.3 to improve UFO filter
capabilities and behavior. These improvements reflect partner requirements for
accommodating intermediate QC data (ObsFunctions) and detailed QC information in
addition to a simple pass/fail flag. The order of execution of UFO filters will be improved and
made clearer to UFO developers. In addition, support for station master lists and observation
filtering based on external runtime data will be implemented.
OBS3.4 provides observation data management for R2D2 (developed in JEDI4). This
includes storage and organisation of observation streams and data sets in IODA, ensuring
that IODA content has accurate metadata, and documentation of the R2D2 observation data
catalogue. OBS3.5 concerns updates to the base functionalities of the IODA, IODA-
converters, UFO, and diagnostics repositories that result from development in OBS1 and
OBS2, particularly during code sprints.
JCSDA Operating Plan 2021_rev 1_sanitized Page 25 of 88DocuSign Envelope ID: C2EA4FAF-FD7C-48AC-8198-0590EDA6C0A0
Task SOCA1: Marine model interfaces to JEDI
Task Description
This task will consist of the continued development of the MOM6 (SOCA1.1) and CICE6
(SOCA1.3) interfaces to JEDI with the addition of ROMS (SOCA1.2) and a machine learning
based sea surface temperature model (SOCA1.4), both already in development. Since
personnel resources are not available for the implementation of a dedicated WWIII interface,
the focus will be on model initialization where the wave analysis will be implemented inside of
an existing interface and solved as a 2D analysis problem. A nonlinear variable transform will
be implemented to convert the 2D significant wave height analysis variable to the 4D wave
spectra field required by WWIII (SOCA1.5). This strategy is currently implemented for sea-
ice DA within the MOM6 interface to JEDI. A deliverable of note is the public release and
documentation of the MOM6 interface to JEDI (SOCA1.1) by the end of the first quarter.
While developing the ROMS interface, and in particular the interface to the tangent model of
ROMS, we will investigate the possibility of its use as a surrogate tangent linear model for
MOM6 (SOCA1.2). This work would be exploratory and used to understand the level of effort
required to implement 4DVAR within a regional MOM6 configuration.
A significant amount of effort is also required to develop and test linear and nonlinear
variable changes for balances and positive definite variables (SOCA1.1).
Task SOCA2: Marine Applications
Task Description
This task supports the implementation of JEDI based applications for the initialization of sub-
seasonal to seasonal forecast systems (S2S) and of the high resolution regional MOM6 used
in the Hurricane Analysis Forecast System (HAFS). The objectives for the 2021 AOP are to
extend the reanalysis development from AOP2020 to the S2S application (SOCA2.10) and to
implement the necessary elements towards a JEDI based DA system for a regional, high-
resolution MOM6 application (SOCA2.11), which will eventually replace HYCOM as the next
generation ocean component of the HAFS.
Maintaining, refactoring and documenting the marine UFO’s will be at the forefront of the
effort for the first quarter, a public release is planned for the middle of the first quarter
(SOCA2.1).
New forward operators are in the planning (SOCA2.2), radial velocity for HF radar and an
interface to a Spectral Irradiance Model, including the development of its tangent and adjoint
model. This later work will rely on the GMAO in kind expertise while the JCSDA will provide
infrastructure guidance and support.
Some level of QA/QC has been implemented during the 2020 AOP to allow for the
JCSDA Operating Plan 2021_rev 1_sanitized Page 26 of 88You can also read
