Leveraging AI for Self-Driving Cars at GM - Efrat Rosenman, Ph.D. Head of Cognitive Driving Group General Motors - Advanced Technical Center, Israel
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Leveraging AI for Self-Driving
Cars at GM
Efrat Rosenman, Ph.D.
Head of Cognitive Driving Group
General Motors – Advanced Technical Center, Israel
Agenda
• The vision
• From ADAS (Advance Driving Assistance Systems) to AV (Autonomous Vehicles)
• AI for Self-Driving cars
• ADAS, AV and in-between
• Summary
10/19/2017 General Motors 2
The Vision
• Mobility – one of the most significant revolutions of modern times
• Self-driving cars will take mobility to a completely new phase…
”Zero Crashes, Zero Emissions, Zero Congestion” (Mary Barra, GM CEO)
?
10/19/2017 General Motors 3
The Vision
Increase Safety Increase Productivity
Increase Mobility: anywhere, anytime Increase Car Sharing & Reduce Road Capacity and Parking needs
10/19/2017 General Motors 4
From ADAS to AV
L5:Full
automation Anywhere, anytime
Level 4: High
Fully autonomous specific scenarios
automation
Level 3: Conditional Highway driving (driver takes
automation control with notice)
Level 2: Partial automation Traffic jam assist
Level 1: Driver assistance Cruise control, lane position
Level 0: Driver in full control Info, warnings
10/19/2017 General Motors 5
From ADAS to AV • Will incremental steps get us to the top of this pyramid? 10/19/2017 General Motors 6
Components of self driving cars
Decision
Sensing Mapping Perception Control
Making
10/19/2017 General Motors 7
Components of self driving cars
AI AGENT serves as the “brain” of the car
Decision
Perception Control
Making
10/19/2017 General Motors 8
AI for Self-Driving Cars
9
AI in Perception
• Unsupervised learning
• Finding structure in point clouds
• Feature learning
• Supervised learning
• Object detection
• 2D object recognition (Classification)
• 3D scene understanding and modeling (3D objects
pose)
• Semantic segmentation (boundaries of objects, free
space)
10/19/2017 General Motors 11AI in Perception - E2E trend
• Classification:
Pixels Key Points SIFT features Model Labels
• Scene understanding:
Pixels Segmentation Object Contextual Scene
detection relations description
• Perception:
Sensors 2D object Depth Pose estimation 3D World state
detection estimation
10/19/2017 General Motors 12AI in Perception - E2E trend
• Classification:
Pixels Key Points DNN
SIFT features Model Labels
• Scene understanding:
Pixels Segmentation DNN
Object Contextual Scene
detection relations description
• Perception:
Sensors 2D object Depth Pose estimation 3D World state
detection
DNN
estimation
10/19/2017 General Motors 13Towards E2E: Sensors Fusion Low Level: raw data High Level: tailored hierarchy combined in input stage between sensors • All sensors • Utilizes domain contribute knowledge • Enables learning • Model is of complex explainable dependencies “optimally” • Sparse Vs. dense • Based on tailored sensors rules • Larger models, • Suboptimal harder to learn performance 10/19/2017 General Motors 14
Towards E2E: Multi-Task Learning
• Most our outputs are inter related
• Objects, free space, lanes, etc.
• Cross regularization allows reaching a better local minima
• TPT
• Major parts of the Deep Net are used for multiple tasks
• Data Efficiency
Mask R-CNN Facebook AI
Research (FAIR); Apr 2017
10/19/2017 General Motors 15What about data?
16Automatic Data Annotation
• Data is the key contributor to perception
accuracy – With no visible saturation
• How can we create annotated data
• Manual annotation – Expensive and inaccurate
• Automatically
Revisiting Unreasonable Effectiveness of
Data in Deep Learning Era, Google 2017
10/19/2017 General Motors 17Automatic Data Annotation
• Technology
• High end sensors (Lidar, IMU, etc.)
• High accuracy detectors (on behalf of computation time)
10/19/2017 General Motors 18Example – AGT for StixelNet
• StixelNet - Monocular obstacle detection
• Based on stixel representation
• Identify road free space
[Badino, Franke, Pfeiffer 2009]
• Ground truthing is based on Lidar Compact, local representation
Dan Levi, Noa Garnett, Ethan Fetaya. StixelNet : A Deep Convolutional Network Lidar (Velodyne HDL32) is used to identify
for Obstacle Detection and Road Segmentation. In BMVC 2015. obstacle on each stixel in the image
10/19/2017 General Motors 19Is Perception “solved”?
• Challenge of Cost
• Sensors
• Mapping
• Computation
• Challenge of false positive & false negative
• Data uncertainty (noise)
• Model uncertainty (confidence)
Label: Cyclist
RGB: Pedestrian (0.56)
10/19/2017 General Motors 20Decision Making
Decision
Perception Control
Making
10/19/2017 General Motors 21Learning Decision Making
Decision Making cannot learn from static examples
Need interactive domain
- > Reinforcement Learning (RL)
RL has seen some major successes in the recent years:
Autonomous Helicopter Poker Atari Go
[Google Deepmind] source: nbcnews
Flight [Bowling et al] source: wikipedia [Google deepmind] source: uk business
insider
[Ng et al] source: ai.stanford.edu
10/19/2017 General Motors 22RL challenges in Self-Driving agents
• Learn to act in a very high dimensional space
• Plan sequences of driving actions
• Predict long term behaviors of other road users
• Few sec
• Complicated situations
• Negotiate with other road user
• Guarantee safety
10/19/2017 General Motors 23Simulation
• Advanced simulations are required
• Multi-agent
• Various conditions
• Focus on “interesting miles”
• Drive billions of “virtual miles” (fuzzing)
Waymo simulation:
https://www.engadget.com/2017/09/11/waymo-self-
driving-car-simulator-intersection/
“Any system that works for self driving cars will be a combination of more than 99
percent simulation.. plus some on-road testing.” [Huei Peng director of Mcity, the
University of Michigan’s autonomous- and connected- vehicle lab]
10/19/2017 General Motors 24Safety Guarantees - From ADAS to AV Will incremental steps get us to the top of this pyramid? The technological heart is different in kind 10/19/2017
What’s the difference? • For ADAS – Safety guarantee is based on the driver • For autonomous – Safety guarantee should come from the system itself 10/19/2017 General Motors 26
Example: Highway Driving in Super Cruise™ The 2018 Cadillac CT6 will feature Super Cruise™ - a hands-free driving technology for the highway It includes an Exclusive driver attention system to support safe operation 10/19/2017 General Motors 27
Safe Driving for level 4/5
• System should handle 100% of the cases
• Redundancy requires at all levels
• Sensing
• Algorithm
• Computing
• Control
• Fallback strategies
• Guarantee of Safety is a must to the acceptance of AV
• Statistical data-driven approach [miles-per-interrupts] requires driving billions of
miles to validate an agent
• Should be repeated with every SW version
• Need safety constrains (rule-based/model-based)
10/19/2017 General Motors 28Summary • Advances in AI are key to success of self- driving cars • AI-based features can bring ADAS to a new level in terms of accidence avoidance, productivity gain and saving in human lives • Level 4/5 AV should be a parallel effort focus on redundancy and safety constrains 10/19/2017 General Motors 29
GM Advanced Technical Center in Israel (ATCI)
Thank you 10/19/2017 General Motors 31
You can also read
