Outcome from Mind-map exercise - ACI 14th-15th March 2018 Andy Ash - Dstl SSA PTA Kent Miller - US EOARD SSA Lead

Page created by Francisco Chavez
 
CONTINUE READING
ACI 14th-15th March 2018
Outcome from Mind-map exercise

Andy Ash – Dstl SSA PTA
Kent Miller – US EOARD SSA Lead

      18 March 2018

      © Crown copyright 2018 Dstl
                                    UK OFFICIAL
Overview

• Following material represents output from the Astrodynamics
  Community of Interest (ACI) #11
   – Focussed on characterisation of objects in Low Earth Orbit (LEO)
     against the defined mission aim (next slide)
   – Present initial ‘mind-map’ to help formulate problem
   – Comments captured from ACI members at the workshop
   – Summary comments from Dstl chair (SSA PTA) based on review of
     ACI views
   – Next steps and schedule
• Aim of this output is to help inform proposals to EOARD and/or
  Dstl R-cloud process
         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
Workshop Objective

     The UK and US have a requirement to identify and to
 characterize space objects and to attribute the status of each
   as ‘active’ or ‘inactive’ with a high degree of confidence.
Traditional approaches to this problem revolve around manual
  analysis of target signature variations (such as light curves
and spectra). Demonstrate an ability to meet this requirement
                     using novel approaches

• Primary focus: LEO with option to consider higher orbital regimes

         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
What is the exam
    question; focus on
                                           Do we have a            What data sources are
                                                                     out there is open
                                                                                                Initial mind-map
                                          sufficient set of
     LEO ISR? What                       classes defined?          source to support this?
   about other regimes?

                                                                                                      Can we assign a ‘value’ to
                                                                               Data                     different data sets to
        Ground truth on                   Threat                                                        assist this problem?
     targets; definition of              Definition
      training versus test
       data sets/ targets
                                                        Space Object                        Finger-printing/
                                                       Characterisation                    unique signatures

                                                                                                                  Identify unique
              Sensor                                                                                              phenomena for
            Observations                                                                                       different s/c classes

                                                                      Data processing               Anomaly detection

   How do we validate                                                                                  Algorithm balance;
results? Can we identify               Possible cooperative                                                confidence,
 target that is due to de-             observation campaign        Analysis of bulk data              timeliness, operator
      commission?                            (5-eyes?)              (TLE, orbits, EO,                    interaction etc.
                                                                        radar etc.)

                       18 March 2018

                       © Crown copyright 2018 Dstl
                                                              UK OFFICIAL
Chair Comments (1)
•   A lot of discussion on threat & classes; active/ inactive likely too coarse,
    more interested in intent and risk
     – Need to articulate set of targets considered risks now and into the future
       (e.g. smaller satellites with deployable antennas)
     – Need to better understand what would cause a satellite with a particular
       mission to be unable to perform its primary function and then try to infer
       based on measurements (e.g. loss of power, ACS, thermal control etc.)
     – This needs to be inherently linked to the wider mission and intended use of
       the information we generate to scale confidence levels to potential
       responsive actions
•   Need to look at Red intentions to fool/ deny Blue SSA systems trying to
    do the above; may act as a indicator in itself

           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
Chair Comments (2)
•   Sensors: ‘old favourites’, open questions on utility of novel approaches
    (e.g. hyperspectral), more about exploitation and fusion
     – Lack of ground truth likely to be big(gest?) challenge in terms of training our
       algorithms
     – Thermal response/ profile would provide greater insight into s/c operation
     – Have an initial list of target features we want to know: orbit, attitude,
       temperature, solar panel orientation, size, mass…
          • Want to know these potentially over long timescales
     – How do we assess the utility, assign a confidence level and check validity of
       sensor data?
•   Clear need for distributed experiments, uncertainty about how best to
    design or execute these (gov vs. civ vs. amateur vs. sensor type….)
     – Connectivity and standards still lacking to enable this

            18 March 2018

            © Crown copyright 2018 Dstl
                                          UK OFFICIAL
Chair Comments (3)
• Have a healthy set of ideas for algorithmic approaches including
  both new and re-purposed existing methods
• Role of simulation unclear:
   – Potential route to mitigate lack of test/ training data but potential risk
     that this will not represent what the threat does in reality
• Growth in well characterised s/c in LEO (high accuracy orbit,
  telemetry and characterised before launch) presents an
  opportunity to test techniques and provide calibration targets ‘on
  the fly’
• Need to understand algorithm scalability and interface with
  analysts as part of wider mission requirements/ execution

         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
Chair Comments (4)
• Metrics:
   – Need to assess/assign value to data generated, but not clear on
     how we do this
   – How do we assess our performance overall? Especially with a lack
     of well characterised/ ground truth test cases?
• Standardisation of data and generation of a database/ historical
  archive of data would enable candidate algorithms
   – Common theme from previous ACIs, but still no clear way to
     implement this
   – Do we have a suggested way forward?

         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
Comments from ACI participants

    18 March 2018

    © Crown copyright 2018 Dstl
                                  UK OFFICIAL
Threat Definition
•   Exam Question:
    – What is bigger picture problem?
    – Want to understand potential threat, i.e. disruption to our future action
      caused by another RSO
    – Need to determine confidence of attribution to enable differing levels of
      response
    – Can a potential adversary perform unwanted ISR ops against our
      operations?
    – Operation threats: Kinetic collision, jamming
    – Intelligence threats: Signals intelligence, visual inspection
    – What space assets can approach our critical assets without us receiving
      sufficient warning to mitigate?
    – What hostile action will be taken to degrade our SSA? Role of game theory
      to examine red vs blue behaviours

           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
Threat Definition

• Size/ capability
   – Trend towards smaller s/c, need to look at future threat
     evolution and make sure any techniques can ‘keep up’
   – Should be interested in any object that might pose a threat
   – How small an object do we need/ can we see (now & in the
     future)?

        18 March 2018

        © Crown copyright 2018 Dstl
                                      UK OFFICIAL
Threat Definition

• Classes
   – Set presented insufficient (alive/ dead), need more granularity
       • Alive, dead, indeterminate, zombie
       • Expansion of classes should be based on observations
   – Should focus on intent rather than class definition
   – Aim to ascertain mission purpose (ISR, Comms etc.) as well as
     status
   – How do we handle/ mitigate lack of ground truth on current status of
     observable objects?
   – Need a common threat taxonomy

         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
Data
•   Sources:
    – Passive
        • Optical (tracks, filtered, imaging, astrometry, photometry)
        • RF (Broadcast activity, Signals INT, bi-static illumination)
    – Active
        • RF (Radar, ranging data, RCS variations, imaging)
    – Want multi-modality data to highlight different aspects of target
    – Want to know mass, thermal properties/ variation, attitude
    – Astro survey systems should be examined
    – University and commercial sensors
    – Open source data? E.g. Mini Megatorora
    – Distributed data repository would enable this work
    – What about space based?

           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
Data

• Value & veracity
   – Open source data needs to be assigned a value for veracity
     to avoid erroneous assumptions persisting
   – Need long term historical data to enable analysis, may be
     hard to retrospectively accumulate
   – “Noise & clutter” – events occurring naturally/ serendipitously
     that mimic the exact behaviour we are trying to observe
   – How do we access potentially higher quality data generated
     by government classified sources?

        18 March 2018

        © Crown copyright 2018 Dstl
                                      UK OFFICIAL
Data

• Ground truth
  – Can we/ how do we generate simulated representations of
    what we expect to observe for such a large range of potential
    circumstances?
  – Is ground truth actually available to enable training of
    techniques? Esp for Machine Learning?
  – Precise orbital data and significant amount of information on
    highly characterised LEO s/c is now becoming available
  – Creation of a single, standardised set of test data would
    enable this work

       18 March 2018

       © Crown copyright 2018 Dstl
                                     UK OFFICIAL
Finger-Printing
• Definition of unique set of features and/ or behaviours:
   – Spacecraft class and mission (ISR, Comms etc.)
   – Materials and associated spectra
   – Motion; orbit, acceleration, vibration, rotation, routine re-orientations
   – Shape; potentially changing (deployments, solar panel motion)
• Need to examine these factors as a function of time for long term
  analysis and detection changes
• Need to work out which of the above are measurabele?
• Standardised data formats required to enable this area
   – Maintain databased on known non-threatening objects to deterine
     new objects that break this pattern

         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
Data Processing
•   Candidates:
    – Need automatic manoeuvre detection; need to work on algorithms for this,
      test on simulations and then real data
        • Are TLEs sufficient for this?
    – Intelligent data fusion using all data available (EO, RF, TLE etc.)
    – Explore access and use to existing US tools in AFRL, DARPA and NASIC
    – Exploit greater number of highly characterised s/c in LEO to perform
      comparative analysis of s/c in same FOV of sensor
    – Methods to extract attitude/ solar panel control, thermal control
    – Use of target simulation to predicted expected sensor responses
    – Anomaly detection; easy at sensor level but hard to infer what this means for
      the observed s/c
        • What constitutes anomalous behaviour? Change in orbit/ state?
        • Predictive analysis much harder!
           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
Data Processing
•   Constraints and issues:
     – Machine Learning require positive example and training data that may be
       limited
         • Hard to predict future threats/ behaviour before crisis
         • Simulation may help but is fraught with issues
     – Need a defined standard to enable variety of processing techniques
     – Need way to ‘weight’ and combine solutions and techniques, but unclear
       how best to achieve this
     – Likely to need much higher knowledge of the nature environment to
       understand its effect on s/c
     – Need to understand scalability of algorithms
     – Need to understand machine/analyst link better
     – Need to understand timeliness driven by wider ops requirements

           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
Observations and Experiments

• Distributed sensor architecture required
   – Need a way to coordinate and optimise observation of the network
   – What is balance between few exquisite sites vs many low cost
     sensors
   – Deploy a high gain wide band RF antennas to do passive 3D
     imaging of LEO s/c
   – Get every nerd (not my word!) to post pictures of a high interest
     event and examine our capacity to ingest and use large data set
   – How do we rigorously assign ‘value’ to each node in the network or
     piece of data? Potential role for information theory

         18 March 2018

         © Crown copyright 2018 Dstl
                                       UK OFFICIAL
Observations and Experiments

• Challenges/ considerations
   – Need an architecture and consider DDS, ICD, standardisation etc.
   – How do we mitigate short arc measurements when trying to perform
     long term pattern of life?
   – Can we extrapolate data from short obs windows?
   – Short exposures vs long exposure/ deep observations?
   – Need to build up a database of real observation data of known/
     similar s/c to help overcome data sparsity
   – Engage NASA to do another well characterised re-entry event
   – Engage NASA/ ESA regarding cooperative targets to aid ground
     truth data generation

        18 March 2018

        © Crown copyright 2018 Dstl
                                      UK OFFICIAL
Next Steps

    18 March 2018

    © Crown copyright 2017 Dstl
                                  UK OFFICIAL
Schedule of activity
•   15th March 2018: Workshop with community and briefs on proposed
    activity of work
•   20th April 2018: Distribution of minutes from the workshop to the
    participants. Deadline for white papers to EOARD.
•   31st May 2018: Expressions of Interest (EoI) and/or proposals
    submitted by Dstl via appropriate contracting mechanism such as R-
    cloud. Joint UK-US review of returns from suppliers.
•   30th June 2018: Feedback provided to suppliers on unsuccessful
    proposals.
•   31st August 2018: Dstl contract award to successful participants. Grant
    proposal agreed and on-contract from EOARD

           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
Proposal/ Bid process

•   Grant proposals sent to US EOARD for consideration via
    (https://www.grants.gov/)
•   R-cloud calls uploaded by Dstl based on workshop outcomes
    (https://rcloud.dstl.gov.uk/)
     – Will advise on capability area, likely under new ‘Space’ or ‘C4ISR’
•   Dstl remit: TRL1-6 // EOARD remit: TRL1-2
•   Joint review of bids and proposals by Dstl and EOARD
•   Scored based on:
     – Technical approach  novel methods favoured over existing
     – Partnering  preference given to those demonstrating UK/US linkages
     – Costs; ROM cost for studies is ~£100k/ $100k

           18 March 2018

           © Crown copyright 2018 Dstl
                                         UK OFFICIAL
You can also read