Equipe Intermedia équipe TIPIC labo CNRS SAMOVAR - Bernadette Dorizzi
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Equipe Intermedia
équipe TIPIC labo CNRS SAMOVAR
Bernadette Dorizzi
http://www.telecom-sudparis.eu/eph
Bernadette.Dorizzi@it-sudparis.eu
Institut MinesTélécom;Télécom SudParis
Institut Mines-Télécom
Intermedia
une équipe bi-localisée EVRY-Nano-INNOV, Palaiseau
■ Projet Bio-Identité:
● Bernadette Dorizzi
● Mounim El Yacoubi
● Sonia Garcia
● Dijana Petrovska
● Doctorants: Mouna Selmi, Mohamed Ibn Kheder, Nadia Othman, Janio Canuto,
Raida Hentati
■ Projet Geste:
● Patrick Horain
● Doctorants: Maher Mkinini,
■ Projet télévigilance
● Jerome Boudy
● Jean louis Baldinger
● Doctorants: Pierrick Mihorat, Toufik Guettari, Mohamed Sehili
● Post doc Paulo Cavalcante
pa
ge Institut Mines-Télécom
2
Research Context
■ Propose and adapt models from signal and image
processing, statistical pattern recognition for various
applications related to interactions between Man and
Machine
■ Scientific background
● Statistical Pattern recognition
● Machine Learning
● Image processing, signal processing
● Sensors and micro-electronics
● Multi-sensor data fusion
■ Models:
● HMM, particular filtering, PSO, entropy, DTW, Gabor
wavelets, ACP, LDA, Appearance and registration models.
pa
ge Institut Mines-Télécom
3
Intermedia : Interactions for Multimedia
7 permanent staff, PhD students, post-doc
■ Application domains:
● Biometrics (including iris, 2D-3D face, speech,
signature)
● Crypto-biometrics (revocable biometrics and crypto-
biometric key generation from biometric data)
● Audio Indexation using the speech recognition
framework (publicity detection, music,…)
● Avatars personalisation
● Action and Activity recognition
● Home healthcare activity
● Gesture-based communication in networked virtual
environments
● Person re-identification in videos
● Video indexing and retrieval through face and silhouette
Institut Mines-Télécom
Intermedia 1/…
Biometrics
Bernadette Dorizzi
Mounim El Yacoubi, Sonia Garcia Dijana Petrovska,
Intermedia/TELECOM SudParis
Authentication of persons from their personal characteristics
(physiological, behavioural)
Institut Mines-Télécom
3D morphable model
From side view to frontal view
Dynamic Signatures
• Design of models relying on continuous HMMs
• Tests on different databases (Philips,
BIOMET, MYCT, SVC2004)
•Participation to the international evaluation
campaigns SVC 2004, BioSecure 2007,
BSEC09
Combining Biometrics and
Cryptography Iris verification from
for Secure Authentication degradated acquisition
conditions
Differential sensors IR for face and Télécom
hand vein verification
SudParis
Interests •2D and 3D face recognition
Multimodal fusion
• Scores Fusion with means models, SVM, décision trees
• Evaluation Protocols on multi-modal databases
Usage Tests and field studies in the
context of biometric systems
deploiement.
pa
ge
6
Research problems
■ Development of new algorithms
● On-line signature (patent)
● Iris verification in degraded mode (patent)
● Face Verification in 2D and 3D
● Quality assessment
■ Multimodality : Development and test of score fusion algorithms,
independance tests, feature selection and fusion (1 PhD)
■ Biometrics and Security
● Development of new crypto-biometric strategies
■ Coupling sensors/algorithms (collaboration Société NIT, Yang Ni)
● Differential image sensor able to decrease illumination effects
■ Assessment protocols for biometric algorithms and multibiometric
algorithms (projet BioSecure)
■ Biometric implementation on embedded systems (PDA, mobile devices)
(projets VINSI, SecurePhone, SIC)
● Taking into account degradations linked to mobility, Interest of multibiometry
Signature verification on iphone, ipad, android platforms
Usage tests and field studies in the framework of deploiement of biometric
systems.
Institut Mines-Télécom
Principaux résultats de recherche
■ Reconnaissance de visage en 2D avec conditions d’illumination
variables par codage de Gabor et projection par LDA
■ Vérification par le visage en mode proche infra-rouge: méthode
d’appariement locale basée sur les points de contour
■ Qualité locale mesurée par GMM pour reconnaissance de l’iris en
mode dégradé
■ Mesure d’entropie pour qualifier des qualités de signatures
manuscrites en-ligne et la texture d’iris
■ Modèles de “Morphing” 2D-3D pour la reconnaissance de visage avec
variations de pose, pour l’animation d’avatars 3D
■ Fusion multimodale de scores, d’images et de caractéristiques
■ Sélection de variables par méthodes “Optimisation par Essai de
Particules”
■ Sécurité des systèmes biométriques (biométrie révocable et crypto-
biométrie) Crypto-biométrie : obtention d’une clefs crypto-
biométrique de 147 bits à partir d’images d’iris et de visage,
proposition de nouveau protocoles
■ Indexation audio en utilisant la principe de reconnaissance de la
parole (détection de publicité, musique,…) excellents résultats sur
données YACAST (diffusions radios) et QUAERO
■ Ré-identification dans des videos par approches SURF et
représentation parcimonieuse
8 Institut Mines-Télécom SSD 2013
On- line signature : Personal Entropy Measure
Nesma
Houmani,
Sonia
Garcia-‐Salice3
■ We proposed a Personal Entropy Measure computed locally on a set
of genuine signatures [1,2,3]
■ We generated with such measure 3 writer categories, coherent in terms
of signatures’ visual aspect, complexity and variability.
■ We tested 3 classifiers on each category of users and compare results
There are users by far more difficult (a factor 2) to recognize than
others: High Personal Entropy category [2,3]
■ We studied in [4] the Robustness of Coordinates, Pen Pressure and Pen inclination angles to Time
Variability with Personal Entropy (x,y) is the most robust combination to long-term time variability
(as observed by other criteria and performance assessment) .
[1] S. Garcia-Salicetti, N. Houmani, B. Dorizzi, "A Client-entropy Measure for On-line Signatures", Proc. of IEEE Biometrics Symposium
(BSYM 2008), Tampa, USA, September 2008.
[2] N. Houmani, S. Garcia-Salicetti, B. Dorizzi, "A Novel Personal Entropy Measure confronted with Online Signature Verification Systems’
Performance", Proc. Of IEEE Second International Conference on Biometrics: Theory, Applications and Systems (BTAS 2008), Washington,
September 2008.
[3] S. Garcia-Salicetti, N. Houmani, B. Dorizzi, "A Novel Criterion for Writer Enrolment based on a Time-Normalized Signature Sample
Entropy Measure", EURASIP Journal on Advances in Signal Processing , vol. 2009, Article ID 964746, 12 pages, 2009. doi:
10.1155/2009/964746.
[4] N. Houmani, S. Garcia-Salicetti, B. Dorizzi, "On assessing the Robustness of Pen Coordinates, Pen Pressure and Pen inclination to Short-
term and Long-term Time Variability with Personal Entropy", Proc. Of IEEE Second International Conference on Biometrics: Theory,
Applications and Systems (BTAS 2009), Washington, September 2009.
Institut Mines-Télécom
Combining Biometrics and Cryptography
for Secure Authentication
Sanjay Kanade, Dijana Petrovska
Problem: Biometric data is not revocable; cannot replace and reissue the template in case of compromise
Solution: Use biometrics in combination with password; both must be provided at the same time for the
system to work; there is no sequential processing
Revocable / Verification Yes/No
Enrollment cancelable
Password Key regeneration Key
template (101010011101010100011
010101010101110101101.
.. …)
Password
Our work:
■ A shuffling scheme is used which makes the biometric enrollment data (template) revocable
■ Different templates can be issued for different applications; So user privacy is preserved
■ Stored data does not reveal information about the biometric enrollment data
■ Reduce variability in biometric data using error correcting codes [2]; improves biometric performance
■ Cryptographic keys having 83-bit entropy in single eye mode and 147-bit entropy in two-eye mode can also be
obtained; this is the highest reported entropy in literature
1. Kanade, S.; Camara, D.; Krichen, E.; Petrovska-Delacrétaz, D. & Dorizzi, B. “Three Factor Scheme for Biometric-Based Cryptographic Key
Regeneration Using Iris”, The 6th Biometrics Symposium 2008 (BSYM2008), 2008
2. Kanade, S.; Petrovska-Delacrétaz, D. & Dorizzi, B., “Cancelable Iris Biometrics and Using Error Correcting Codes to Reduce Variability in Biometric
Data”, IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009
3. Kanade, S.; Petrovska-Delacrétaz, D. & Dorizzi, B., “Multi-biometrics based cryptographic key regeneration scheme”, IEEE Conference on Biometrics:
Theory, applications and systems (BTAS), 2009
Institut Mines-Télécom 10
Gait Recognition
Mounim El Yacoubi, Ayet Shaiek, Mohamed Ibn-Kheder,
■ Silhouette Normalization
● Normalizing the varying distance from each gait
frame to camera
■ Gait Period Detection
● Analysis of the Density of Foreground Pixels
associated with the legs
■ Feature Extraction
● Vector of the Widths (Structural-Dynamic
Information)
● Motion Vector between 2 Consecutive Frames
(explicit dynamic information)
■ Recognition
● Hidden Markov Models (HMMs)
− Explicitly divide the gait sequences into its periods
− Robustness toward Gait sequence local variations
● Dynamic selection of the feature type based on:
− Kmeans distance
− PCA output
− Output of a generic HMM
Cf. Ayet SHAIEK, “Reconnaissance des personnes à
partir de la démarche, “ Master Report, June
2009
Page 11 Institut Mines-TélécomPerson Re-identification System based on
SURF Matching (M El Yacoubi, M Ibn Kheder, B Dorizzi)
Detection of the
Test Video Interest Region
Motion and
Silhouette Detection
Interest Points (SURF)
Extraction
Majority Vote Interest
Rule Points Reference
Matching Database of
SURFs
12
17 Institut Mines-Télécom SSD 2013Experimental Results
" Re-identification Performance = Correct Classification Rate (CCR)
Re-Identification = f(Angle Difference)
Re-Identification = f(Angle Pairs)
CASIA Dataset-A : 20 persons
6 Different View Angles
2 Sequences for each View Angle
1 sequence as Reference
1 Sequence for Test
13 Institut Mines-Télécom SSD 2013SURF with SPARSE representation/
Results on PRID-2011
PRID-2011:
749 persons as Reference.
200 persons for Test.
In average,100 images per person.
Approach Re-identification
rate
Combination of 19.18%
two methods [18]
Probabilistic 22%
SURF Matching
[25]
Our approach with 28%
sparsity
CMC curve
14 Institut Mines-Télécom SSD 2013Human Activity Recognition in videos
Mouna Selmi, Mounim El Yacoubi, Bernadette Dorizzi
Modelization:
Project Juliette (Feder FP7) • Video = Activities sequences
Partners : Telecom SudParis, Aldebaran • Activity = Sequences of elementary actions
Robotics, Brain Vision Systems (BVS), Features: spatio-temporal interest points
Institut de la vision Optic flow
■ Compagnon Robot at home for restricted Hierachical Classification
mobility persons Probabilistic Model: Conditional Random Fields
● Old, visually disabled, … (CRFs)
● Acquisition of video sequences of humanLight integration of features of different natures
activities
■ Challenge
● Simultaneous segmentation and
Recognition
Page 15 Institut Mines-TélécomBiometric Multimodal Fusion
Lorène Allano, R. Raghavendra, Bernadette Dorizzi
■ Score Fusion Scheme, dependency measures,
building of efficient virtual databases
■ Feature and Image Fusion schemes, Particle
Swarm Optimization feature selection
•Lorene Allano, Bernadette Dorizzi , Sonia Garcia-Salicetti, « Tuning Cost and Performance in Multi-Biometric Systems: A Novel and
Consistent View of Fusion Strategies based on the Sequential Probability Ratio Test (SPRT)” PRL (2010)
•Raghavendra.R, Bernadette Dorizzi, et al.“Designing Efficient Fusion Schemes for Multimodal Biometric System using Face and
Palmprint” submitted PR (2009)
Institut Mines-Télécom3D static model for pose transformation
(Dijana Petrovska, D. Zhou)
Original with landmarks Result
Institut Mines-Télécom
17Avatar personalisation with a 3D model
from a 2D image 2
(Dijana Petrovska, D. Zhou)
3D Scans Database for
Building the Morphable Model 3DMM Face Space
High Resolution
(75k vertices)
Face Analyzer
3D Photo-Realistic
2D Input Image
Representation
18Iris verification in degraded mode
Good quality iris image
Constrained acquisition conditions
Bad quality iris images
Less Constrained acquisition conditions
« iris on the move »
19 Institut Mines-Télécom SSD 2013Global system for iris recognition
Segmentation
Normalization
0100001100010011 Encoding 1110111111000101
Match ?
20 Institut Mines-Télécom SSD 2013Recent Results in the bio-identity project
■ PhD Emine Krichen/patent 2007:
● Definition of local quality measure relying on a GMM for
estimating “good quality” iris texture
■ Thales collaboration (2008-2013) : 2 CIFRE PhD
● S. Cremer, B. Dorizzi, S. Garcia, and N. Lemperiere. How a local
quality measure can help improving iris recognition. BIOSIG,
2012.
● T. Lefevre, B. Dorizzi, S. Garcia, and N. Lemperiere and S.
Belardi, Effective Elliptic Fitting For Iris Normalization, Computer
Vision and Image Understanding / parametric active contour
● T. Lefevre, B. Dorizzi, S. Garcia, and N. Lemperiere and S.
Belardi,New Segmentation Quality Metrics for Iris Recognition,
submitted to ICIAR 2013
21 Institut Mines-Télécom SSD 2013Iris Segmentation via Triplet Markov Tree
Collaboration with Wojciech Pieczynski: TSP/CITI
■ Unsupervised eye image segmentation via Triplet
Markov Trees as a first processing for subsequent
iris segmentation
Triplet Markov Tree (Column 3)
Improves segmentation quality versus
Hidden Markov Tree (Column 2)
Two different localizations of the normalization circles for
the same eye image: wrong (grey-level pixel based) on the
left; good (TMF-based) on the right.
Dalila Benboudjema, Nadia Othman,Bernadette Dorizzi, Wojciech Pieczynski, “Challenging eye
segmentation using triplet markov spacial Models”, accepted at ICASSP 2013
22 Institut Mines-Télécom SSD 2013Improving Video-Based Iris Recognition Via
Local Quality Weighted Super Resolution
Nadia Othman, Nesma Houmani, Bernadette Dorizzi, ICPRAM 2013
■ Iris recognition at a distance and on the move: Video
Uncontrolled acquisition: Less constraints
.
Loss in quality
Eye localization Iris extraction
Lack of resolution, blur.
Our proposal:
Strong occlusions: eyelids, eyelashes, Fuse the frames in the video
spots… to get more information from
the person
23 Institut Mines-Télécom SSD 2013Spoofing detection via OCT approaches
Projet PARADE; Collaboration Yanneck Gottesman TSP/EPH
2 pending patents, August 2012
Context: fingerprint recognition
Actual fingerprint systems: due to the 2D surface
acquisition modes, detection of spoofing is very difficult
(death finger, false finger, overlay)
The quality of the recorded image presents a significant
variability due to external parameters (temperature,
humidity, pressure)
Our proposal:
■ Development of a new type of sensor (3D OCT imaging
sub-cutaneous) to test the living character of the
biological tissue presented to the sensor
■ Signal processing / 3D volumes to allow biometric
recognition and detect attempted identity theft
24 Institut Mines-Télécom SSD 2013Finger with an overlay
■ Overlay depth < dimensions groove (sillon)
Without overlay With overlay
Difficult because the overlay depth is inferior to the OCT resolution
Detection is possible as we use the phase image
25 Institut Mines-Télécom SSD 2013Recent Projects
■ BioSecure : Biometric Secure Authentication
● Coordination of the NoE, 2004-2007, 30 partners, 3 M€
● Framework for test of biometric algorithms
■ Biotyful : BIOmetrics and crypTographY for Fair aUthentication
Licensing (2007-2010)
● ANR telecom, ATMEL, FRANCE TELECOM, GET/INT, GREYC
● Cryptobiométrie
VIDEO-SURVEILLANCE
■ Vidéo-ID : Identification via face and Iris in video-surveillance
● ANR CSOSG (2008-2011)
■ Kivaou : Face identification in video-surveillance
● ANR CSOSG (2008-2009)
■ Xvision: Special vision sensors for outdoor applications
● System@tic (2008-2010)
■ METHODEO:
● ANR CSOSG (2011-2013)
■ Juliette: FEDER 2010
Institut Mines-TélécomRecent Projects
■ Nouveau projet ITEA2 PRIBIOSEC
● crypto-biométrie, coordinateur français CASSIDIAN
et PME Secure-IC
■ Collaboration avec PME E-Closing
● Sur la signature électronique avec biométrie
■ Projet SurfOnHertz avec YACAST:
● indexation audio
■ Projet avec PW Consultants:
Institut Mines-TélécomIntermedia 2/.. Institut Mines-Télécom
■ « Télévigilance » or remote Healthcare for Patients at Home :
● for elderly persons (vigilance problems) or persons with cardio-vascular
pathologies or chronic diseases needing remote healtcare and
surveillance
● makes the Patient more secure for his/her health and keeps social link
through connected communication services
● also releases Home Hospitalisation load
■ System connected to a remote centralised Healthcare services
platform :
● mobile terminal fixed to the patient in a full ambulatory way
● connected to remote Healthcare servers
● alarm automatisation at the patient ’s level and management of
Patient’s data at the server level
● merged in the framework of the Smart home with domotic sensors with
potential link to assistance robots (Companion type).
Institut Mines-Télécom■ Multi-sensors mobile terminal recording actimetric and vital
patient ’s data :
● Simplicity of use and robustness to patient deambulation and
movements
● Sensors design towards acceptance (miniaturisation), autonomy
(battery) and noise-robustness (low-complex/embedded efficient
signal processing)
■ Alarm automatisation based of multimodal fusion
● heterogeneous fusion schemes applied on actimetric and
physiological data to control specificity of the distress situations
detection of the whole system [Medjahed-2010]
● forward alarms to remote healthcare center with sufficient information
for remote emergency diagnosis
● Multimodal localisation with internal domotic sensors for patient’s
localisation or ADL [Guettari-2010]
Institut Mines-Télécompa ge Hamid MEDJAHED 31 31
Televigilance Lab at Telecom SudParis
for technical validation
Pa
ge Institut Mines-Télécom
32Televigilance terminal (projet QuoVADIS)
Ambulatory terminal of televigilance designed at Telecom SudParis
and integrated in partnership with ASICA. (Patent J.L. Baldinger)
Fall detection, posture, mouvement, robust on-line pulse, with
emergency push
On-going validation in an hospital context.
Institut Mines-Télécom■ Projet ACI-Ville: MEDIVILLE (2001-03)
■ 3 Projets RNTS et ANR-TecSan:
● TelePat (2004-06): Televigilance médicale
● Tandem (2005-09): Thérapie cognitive et sécurité à domicile
● QuoVadis (2008-10): Sécurité multimodale (capteurs sur la
personne, domotiques et robot)
■ 2 Projets de coopération internationale (échanges):
● SAFETI-IE4IL (2007-10) : France Afrique du Sud
● Brancusi (2009-11) : France et Roumanie
■ Projet IST-FP7 CompanionAble: Combination of the Smart
home and the Robot-Compagnon (2008-11)
Institut Mines-TélécomPerspectives à moyen-/long-terme s’inscrivant dans
les TIC Santé et la stratégie de Telecom SudParis
■ Télévigilance Médicale appelée à faire partie d’applications
plus générales comme la domotique et l’assistance robotique
déjà appelées à se combiner
=> RobotCompanion dialoguant avec son maitre (le patient) mais
aussi veillant à son confort et sa sécurité grâce à une
communication généralisée entre capteurs et modalités de tout
type
■ Combinaison/ Fusion/ Adaptation des données/ paramètres
domotique/ patient par des approches de type Fusion/ Fouille
de données et Context Awareness adaptation
=> Intelligence Ambiante et Nomade
■ Axes de recherche importants pour TSP et IT sur l’application
des TIC, des capteurs et de l’informatique réseau à la Santé
=> engendre des projets transversaux intra- et inter-écoles faisant
appel à des compétences variées et très techniques
Institut Mines-TélécomIntermedia 3/..
Perceiving and rendering users
in a 3D interaction
P. Horain
Institut Mines-Télécom3D body motion capture
by real-time computer vision
■ Motion capture by computer vision
■ Real-time remote virtual rendering
MPEG 4 /BAP
3D/2D registration
Institut Mines-TélécomGesture statistical modeling
Conversational gesture generated with
the Greta software with varying expressivity
Institut Mines-TélécomStatistical gesture models
for 3D motion capture and rendering
Rendering motion parameters
that cannot be captured
WITHOUT gesture model
Hands !
WITH gesture model
Institut Mines-TélécomGPUCV: GPU acceleration for
Computational Vision
→ http://picoforge.int-evry.fr/projects/gpucv
Institut Mines-TélécomParis MobileMii
Evry Platform for the
Development of Nomadic
Saclay Services
Institut Mines-TélécomMulti scale and nomadic Services
Joint platfom partnership
• CEA LIST
• Institut Telecom / Telecom SudParis
Supporting partners
• Academics, end users, industry, clusters (pôle de compétitivité)
MobileMii
• A technology research and development Project
• An infrastructure including an apartment located on the Saclay Campus
Objective and concept :
• Provide prevention, comfort and awareness services to people
• Provide seamless services in the apartment, on the campus and beyond
• Develop hardware and software modules for different applications in relation to the
apartment, the building, the campus
Applications
• Security
• Education
• Energy
• Transportation
• Health
42 Institut Mines-Télécom 12MobileMii:
MobileMiiInfrastructure
: Infrastructure
MobileMii – Saclay:
•A 250m2 modular architecture
•A showroom
•Several technical labs for development and
assessments Saclay Campus
•Situated inside Nano-INNOV building sensors distributed over the
campus
43 Institut Mines-Télécom 12Functionalities being studied
to build services
Sensors /Embedded signal People localization
Vision
processing Gesture recognition
Emotion understanding Equipe
Biometrics for people Gait Analysis INTERMEDIA
identification Data Fusion
Equipe Context awareness
Navigation Geolocalization • Automatic learning
•Ontology Équipe Simbad
•Fusion of heterogeneous sensor
information
Sensorial Man Machine
Interface: MMI Equipe
Middleware Marge
Equipe
S3
Communication Environmental
•Sensor network and communication between sensors equipments
•Management of communication between people • Acquisition and Monitoring
•Coverage of mobility communications (Heat, Air conditioning, intrusion,
fire)
44 Institut Mines-Télécom 12Scenarios, Applicative domains
■ Televigilance: surveillance of dependent people at home
● Sensor and data
− Actimetry (accelerometer, gyroscope ...)
− Monitoring vital signals (heart ...)
● Objectives
− Preventing crisis situations (falls, respiratory, cardiac problems)
− Detection of crisis situations (falls, respiratory, cardiac problems)
− Detection of abnormal behaviors (eg Alzheimer's, stress, lack of food, deshydration)
■ Monitoring the activities of daily living
● Televigilance without embedded sensors (camera, sound ...)
● At home or at work
● Detection of risk situations
■ Security and Global Monitoring of public places
● Detection of abnormal behavior
● Detection of risk situations
● Identification of persons (Biometrics)
■ Intelligent Energy
● In a house
− Managing the various sources of energy (production and storage)
− Conserving energy (heating) based on the presence of people
− Managing peak periods: subscription and especially under-exploitation of global network
● In a public / place to live / working place/ building
− User-based energy saving: lights, computers, appliances
■ Home Automation
− Home or workplace, monitoring of blinds, lights, temperature …
■ Media Room
● New GUI (via tactile sensors)
45 Institut Mines-Télécom 12You can also read
