Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors

Page created by Byron Baker
 
CONTINUE READING
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Embedded Algorithms for Motion
Detection and Processing
28th February 2018

Marco Castellano

System On Sensor Team Leader
Analog, Mems & Sensors
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Embedding Algorithms on Sensors                         2

Existing Embedded                              Standard Solution
     Solutions
                                                         Algo1..2..3..
    Sensor
                                            Sensor       General
                                                         Purpose
   Algo1..2..3..
                                                         Microcontroller

               Pros - Cons                 Pros - Cons
      Low Power                 × (Low) Power
      Number of dice            × Number of dice
      1 communication bus       × 2 communication bus
     × Number of Algos            Number of Algos
     × Configurability            Configurability
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Conclusions                3

Innovative Embedded
      Solution
 Inertial Sensor                 Two digital configurable
 LSM6DSOX                        embedded modules for fast and
                                 effective implementation of
   FSM           MLP
                                 motion detection processing have
  ALG1-16      ALG17-24          been presented.
                                          • A

                   Pros - Cons   Common application cases
                                 shows 20 to 100 times current
         
         1 Ultra Low Power
                                 consumption reduction respect to
          Number of dice
                                 standard solutions.
          1 communication bus
          Number of Algos
          Configurability
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Motion Detection Algorithms                  4

                Inertial algorithms overview

       Glance                                     Carry
                          Fitness
                                      Activity    Pos
Face                                   Rec
 Up/             Wake
Down              Up

                                                 Vibration
          Tap                                    Monitor
          Tap                         Step
Free
                    6D                Count
Fall
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Motion Detection Algorithms                  5

                Inertial algorithms overview

       Glance                                     Carry
                          Fitness
                                      Activity    Pos
Face                                   Rec
 Up/             Wake
Down              Up

                                                 Vibration
          Tap                                    Monitor
          Tap                         Step
Free
                    6D                Count
Fall
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Two Motions Comparison                                            6

               Face-Down Transition                                     Foot Impact

                                                 Medium                                               Medium
                                                 Strong                                               Strong
                                                 Slow                                                 Slow

Facing down the phone determines a sign change on Z       Foot impacts determine peaks, which shape, magnitude
component of acceleration (1 g -> -1g)                    and frequency are heavily variable
Embedded Algorithms for Motion Detection and Processing - Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors
Two Motions Comparison                     7

Face Down Peak to Peak Occurences   Foot Impact Peak to Peak Occurences

              Narrow                             Broad
Motion Detection Algorithms                     8

                Inertial algorithms overview

       DEDUCTIVE                            INDUCTIVE

       Glance                                        Carry
                          Fitness
                                      Activity       Pos
Face                                   Rec
 Up/             Wake
Down              Up

                                                    Vibration
          Tap                                       Monitor
          Tap                         Step
Free
                    6D                Count
Fall
Inductive or Deductive?                          9

                               Divide et impera

                        DEDUCTIVE         INDUCTIVE

• Based on general hypothesis that        • Based on generalization from specific
  predict the observation with high         observations
  precision
                                          • Campaigns of data collecting needed
• Robust and fully validated hypothesis
  needed                                  • The algo is dependant on the
                                            population under consideration
                                            (gender, age, provenience...).
Motion Detection Algorithms                         10

                   Inertial algorithms overview

Finite State Machine For motion        Machine Learning Processing
Detection

          Glance                                             Carry
                                  Fitness
                                              Activity       Pos
 Face                                          Rec
  Up/                Wake
 Down                 Up

                                                            Vibration
             Tap                                            Monitor
             Tap                              Step
 Free
                        6D                    Count
 Fall
Motion Detection Finite State Machine                                              11

               LSM6DSOX is provided with an interpreter that decodes and executes up to 16
                                                         independent Finite State Machines
  Each FSM is intended to detect a single specific gesture.

• Wrist Tilt
• Free Fall
• Pick Up
• Wake-Up                                                          Easy and Effective Application
• Shake                                                                   Development
• Glance
                                                              • STM Library of gestures available
• Tap
                                                              • High level of parametrization
• Motion /Stationary
• Etc…
                                                              • STM development GUI
                                                              • High level of customization
Single Finite State Machine Strategy                    12

         Motion Detection in Three Steps

 Sensors             States           Motion
  Data                               Detection

 Accelerometer        Conditions           Interrupts

  Gyroscope           Commands              Source

   External
                      Parameters      Long Counter
   Sensor
Finite State Machine Inputs                                    13

         Select Appropriate Sensor Data
                for the Application

Sensors                    A wide Set of Inputs to be chosen
 Data
                  1. Accelerometer [ax ay az av ]
                  2. Gyroscope [gx gy gz gv ]
                  3. Calibrated magnetometer [mx my mz mv ]
                  4. Filtered signal #1 [fx fy fz fv ]         Magnitude Available
Accelerometer
                  5. Filtered signal #2 [fx fy fz fv ]
                  6. Filtered signal #3 [fx fy fz fv ]
 Gyroscope
                  7. Filtered signal #4 [fx fy fz fv ]
  External        8. Integrated gyroscope [dx dy dz dv ]
  Sensor
Finite State Machine Core (1)                                14

                                        Smart Memory Allocation

                                                                                       FIXED DATA
      FSM Programs Variables and Instructions                      States               SECTION
A simple program is made by:

• a data section, composed of a fixed part (same size                                 VARIABLE DATA
  for all the FSMs), and a variable part (size is specific for                          SECTION
  each FSM)
                                                                 Fixed data Section
• an instructions section, composed of commands and
  conditions (size is specific for each FSM)                       Variable Data       ISTRUCTION
                                                                      Section            SECTION
                                                                   Instructions
                                                                     Section
Finite State Machine Core (2)                        15

                                    Highly Configurable Core
                                                                           START
                 Instructions Set                          States
Commands are immediately evaluated.                                     RESET   NEXT
When a command is executed, the program pointer
is set to next command/condition line.                                  RESET   NEXT

Conditions are executed at data input arrival.                           COMMAND
If RESET is true the program pointer is set to last        Conditions   PARAMETERS
configured Reset Point;
If NEXT is true the program pointer is set to next line.
                                                           Commands     RESET   NEXT
Parameters are application mask, threshold and
timers                                                     Parameters      CONT
Motion Detected: Status Outputs                        16

    High Reconfigurability on Interrupts Routing:

          Interrupts and Status                    Motion
Interrupts available for each state machine can   Detection
be routed on INT1/INT2 pin

Interrupts can be latched/pulsed.

Source registers available for each state            Interrupts
machine keeps axis positive/negative triggering
the interrupt event
                                                      Source
Long counter countes an event in a FSM
                                                   Long Counter
Example: Build a Simple Effective Face Up to
                                                                                                   17
                         Face Down Detection (1)
          Application case: rotate phone on the table up to down to silence a call

Start: Phone on the table                                           Application Definition

                                                                1) Be still, zeta axis up

                                                 Threshold1     2) Start transition

                                                                3) Check turning zeta respecting
                                       GNTH1
                                                                   timer
                                                 Threshold2
                                                                4) Debounce

                                                - Threshold1    5) Be still, zeta axis down

                                          End: Phone reversed
                               Timer
Example: Build a Simple Effective Face Up to
                                                                                     18
                         Face Down Detection (2)
          Application case: rotate phone on the table up to down to silence a call

Start: Phone on the table

                                                 Threshold1

                                       GNTH1
                                                 Threshold2

                                                - Threshold1

                                          End: Phone reversed
                               Timer
Finite State Machine Conditions                                                                 19

                        States are defined by means of RESET / NEXT conditions
CONDITION               DESCRIPTION                                    CONDITION        DESCRIPTION
0x0        NOP          None execution - condition on current sample   0xB   GRTH1      Any triggered axis > -THRS1
0x1(234)   TI1(234)     Timeout 1(234) expired                         0xC   LRTH1      Any triggered axis  THRS1                     0xD   PZC        Any triggered axis crossed zero pos. slope
0x7(8)     LNTH1(2)     Any triggered axis  THRS1                     0xF   MLPCHK     MLP check
0xA        LLTH1        All triggered axis
Finite State Machine Commands                                20

                          Commands Overview

                                  STOP
                                                    Commands are immediatly
     STHR1(2)                                     evaluated (until a State is found)
                                         CONT
  SELTHR1(3)                    JUMP

                                 SRP     CRP

                                                            Thresholds
    SELMA
                                       UMSKIT
SELMB                                                          Mask
    SM(ABC)                       MSKIT
 SELMC                                                        Timers
                                        MSKITEQ
                                                             Execution
        STIMER3(4)
                                CANGLE
                                                              Output
        STCT0(1)                        REL
                                                              Various
                               SINMUX
Machine Learning Processing                                    21

                         An interpreter decodes and executes up to 8 independent decision trees

     Each Application is intended to detect user contexts.

• Activity recognition
• Fitness activities
• Motion intensity
• Vibration intensity                                             Easy and Effective Application
• Carry position                                                         Development
• Context awareness
                                                             • STM Library for context awareness
• False positive rejection
                                                             • Fully reconfigurable
• Etc…
                                                             • STM development GUI
                                                             • Designed to be used with machine
                                                               learning tool
Machine Learning Approach                                                 22

  Define Classes to be
  recognized

Self learning approach                                     Collecting multiple Logs

Configure the device and
run the application.                                                   Define Features that better
                                                                       characterized the defined
                                                                       classes;

                                • Define Decision Tree using
                                  machine learning tools
Machine Learning Processing Strategy                   23

   Context Awareness defined in Three Steps

   Sensors         Computation      Decision
    Data             Block            Tree

   Accelerometer        Filters      Connections

    Gyroscope          Features      Meta-classifier

     External
                     Connections       Results
     Sensor
Motion Processing Inputs                24

Select Appropriate Sensor Data and Filtering
            for the Application

Sensors                 A wide Set of Inputs to be chosen
 Data
                1. Accelerometer [ax ay az av ] , [av2]
                2. Gyroscope [gx gy gz gv ] , [gv2]
                3. External sensor [mx my mz mv ], [mv2]
Accelerometer

 Gyroscope

  External
  Sensor
Motion Detection Processing(1)                   25

                                      Filters: Smart Memory Allocation

Inputs to Computationals Block are Filtered Data          Computation
                                                            Block
1. High Pass [fx fy fz fv ] , [fv2]
2. Band Pass [fx fy fz fv ] , [fv2]                                       FIXED DATA
3. First Order IIR [fx fy fz fv ] , [fv2]                                  SECTION
4. Second Order IIR [fx fy fz fv ] , [fv2]                    Filters

                                                                         VARIABLE DATA
                                                                           SECTION
Motion Detection Processing(2)                     26

                       Features: Smart Memory Allocation
         Filtered Data is Elaborated:
                                              Computation
              @TIME WINDOW
                                                Block
    @ Trigger (to be configured as feature)
•    Mean
•    Variance
•    Energy                                                    FIXED DATA
•    Peak                                                       SECTION
•    Zero crossings                              Features
•    Min
•    Max                                                      VARIABLE DATA
•    Duration                                   Connections     SECTION
•    Etc.
Motion Detection Decision Trees (1)                                             27

            Outputs from Computation Blocks are Inputs to
                           Decision Trees

Fully Reconfigurable Decision Trees Topology
                                                             Decision
   Tree Example                                               Trees
                                  Typical node
                Start   Input                    Condition

         node
                                                                            Topology
                                                              Connections
                                                                            Connections
                                                                            define
  node
                           True Path      False Path                        the algorithm
                        aaaa
                                                                            behaviour
Motion Detection Decision Trees (2)                                   28

                     Decision Tree Outputs Can Be Filtered

Up to 8 decision trees, Up to 8 Meta-classifier for 8
          Decision Tree Family Results                  Decision
                                                         Trees
     •   A Meta-classifier is a outliers filter [M,N]

     Example: [3,4] meta-classifier
                                                                           Result
IN       a a   a b a      b   b   b
                                                                          Interrupts and
Ca       1 2   3 2 3      2   1   0                                       Status
Cb       0 0   0 1 0      1   2   3                     Meta-classifier   When a change
OUT      x x   a a a      a   a   b                                       of result is
                                                          Results         detected
You can also read