Edge AI-First Approach - Terry Torng Co-Founder - LETI Innovation Days
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Edge AI-First Approach
Terry Torng
Co-Founder
© 2017-2019 Gyrfalcon Technology, Inc. 1
Founded in 2017 By AI Innovators, Entrepreneurs
• 3 AI Accelerator chips in production • 100% Proprietary Technology
• Ecosystem of chips, systems & tools • 70+ patents awarded & in-process
Awards Won
Tier One Analysts 33 Articles in the Press….
Company Offices https://www.gyrfalcontech.ai/newsroom/in-the-press/
Silicon Valley
HQ, Est. 2017
Kyoto
Seoul
Beijing, Shenzhen
© 2017-2019 Gyrfalcon Technology, Inc.
Challenges for the Edge AI
Power consumption
Data
Application management
specific and data
flow control
Algorithm and model
© 2017-2019 Gyrfalcon Technology, Inc.
Hardware, Software and Memory Co-Deisgn
1. NVM Size
Power
2. Matrix
processing
architecture
Intelligence
3. Processing in level
memory
Flexible performance
Application specific
© 2017-2019 Gyrfalcon Technology, Inc. 4
The GTI Accelerator Technology
AI Processed in memory for high performance with low energy and low cost…. Product Offerings
GTI Accelerator Architecture IP
Licensing
MPE (Matrix Processing Engine) using APiM
APiM (AI Processing in Memory) Architecture:
Discrete Chip Portfolio
42 x 42 APiMs = 1 MPE
Parameter Memory 16 MPEs in 280X = 28,224 total MACs &
Board Systems
ALU
ALU Memory
APiM APiM Parameter
APiM
Memory
Data Flow
Key Technology Advantages
Data Flow CTRL
Memory
Control APiM APiM MAC APiM
Data
APiM Unit
High Low Low
Performance Energy Use Chip Cost
APiM APiM APiM
Flexible Chip Size
Interface- AXI
© 2017-2019 Gyrfalcon Technology, Inc. P. 5
GTI – Product Portfolio
High Performance IoT
2801 General Edge 2802 2803 Premium Edge & Cloud
s Devices s MRAM: Non-Volatile Memory s Customization capability to
Chips: Mobile devices, tablets,
Memory, Non-Leakage Power,
optimize parallel processing
Multiple Models, Custom Die
PC per customer specs for
Sizes
Gen. Office products & smart Gen. Gen. superior AI
1 home appliances 2 2
PlaiPlug™ Connect PlaiFi™ Connect
Products: USB3.0 compatible Development
asset for developers & academic
Accessory that enables Edge AI
consumer experiences on
audiences. Ideal for device “proof Android & IOS devices via WiFi.
of concept” and data model Unlocks new capabilities for 3rd
creation projects. party apps.
M.2 Accelerator Card
Boards: NoteBooks, Robots, Autonomous
Evaluation Kit Vehicles
ARM® six-core 64-bit processor w/2.0GHz PCIe Accelerator Card
For Software Development & Prototyping Server Board for Data Centers
--------------------------------- Development Platforms -----------------------
“One Click” GTI NN 4D’s Advanced
Data Model GTI Full Stack Model Data Model
Training Dev. SDK Dev. API Transfer Training Platform
Platform Kits Tools
© 2017-2019 Gyrfalcon Technology, Inc.
Hardware/Software/MRAM co-design
Typical Embedded MRAM AI GTI’s GME Engine Design
Design
Other eMRAM RRAM (R&D) GTI’S eMRAM SRAM
Processing Core Processing Core
w/ AI engine No Leakage power Yes Yes Yes Very High
w/ AI engine
MRAM peripheral Cell Size Small Small Small Very Big
No Size penalty for
No No Yes No
distributed memory
Endurance E6 to E12 E5 E9 to E15 E15
MRAM MRAM NVM Yes Yes Yes No
Peripheral + Array Array Latency High High Low Low
Dynamic power High High Low Low
© 2017-2019 Gyrfalcon Technology, Inc.Industry’s 1st Production AI Chip
(Lightspeeur®2802M)
With Embedded MRAM….
The GME™
Additional Specifications: (Gyrfalcon MRAM Engine)
• 9.9 TOPS/W
• 22nm ASIC
• 20-50% power savings (SRAM, “other
MRAM”)
Customization Options for Large Scale
Customers:
• Up to 10 ns Read Speed (~30 TOPS/W)
• Non-Power Leakage
• Supports multiple models on single chip
• Flexible intelligence level
© 2017-2019 Gyrfalcon Technology, Inc. 8Lightspeeur®2802M
** 40 MB MRAM embedded (or more…)
** Industrial one and only, local no cloud ;
a. Voice ID, voice commands, facial recognition and image classification ----- single chip
b. Ensemble multiple models on single chip to increase accuracy
c. Two 12 bits models on single chip
d. FC layer inside?
e. More?
Autonomous vehicles, robots, AIoT…..
© 2017-2019 Gyrfalcon Technology, Inc. 9Video Content Taking Over Global Data Traffic
Creates Opportunities AND Challenges……
Global Internet Video By Sub-segment
Exabytes/Month, 33% CAGR 2017-2022
By 2021 Almost 70% of
Exabytes/Month All Data Traffic Will Be Video*
33% CAGR 2017-2022
Industry “Pain Points”
From Uncaptured Full Value of Video Content:
1. Can’t search images within video
2. Existing video lacks content labels
© 2017-2019 Gyrfalcon Technology, Inc.Video “Pain Points” Create
New Industry Opportunities
Visual “Search” for Specific Content Video “Data Mining/Retrieval”
1. Search based on picture not descriptive 1. Converts existing video to video with
text labeled content
2. Images used to search existing video 2. Labeled content becomes searchable
files for matching content within each frame
• Superior user experiences
• New business models
© 2017-2019 Gyrfalcon Technology, Inc. 11Until Now, No Solutions…..
No Current Industry Standards Tremendous Computing Efforts
• For Video Data Mining/Retrieval • Expensive computing resources
• Image Searching in Video • Manual labor intensive
© 2017-2019 Gyrfalcon Technology, Inc. 12Performance Comparison Proof Points
CDVA Standard GTI CDVA
PRECISE
Input Size 640x480 640x480
CNN Model Format Floating-point VGG-16 Fixed-point VGG-16 &
CNN Model Size 59M byte 4M byte SMALLER MODEL
Output Vector (Descriptor) Size 512 byte 512 byte
SIZE
Mean Avg Precision Score 86.81% 88.95%
UP TO
Pre- CNN (ms) Extract vector Total (ms)
UP TO 16x processing (ms)
FASTER (ms)
1.9x EXTRACTION GTI CDVA 80 ms 88 ms 15 ms 190 ms
FASTER TIME CDVA Standard 260 ms 78 ms 15 ms 353 ms
EXTRACTION THAN (GPU)
TIME CPU CDVA Standard 260 ms 2800 ms 15 ms 3075 ms
THAN (CPU)
GPU
© 2017-2019 Gyrfalcon Technology, Inc. 13ENHANCE YOUR SEARCH EXPERIENCES USING VISUAL SEARCH
Search Query Search Results
(from video archives, service
providers, database…)
Visual Search
Standard
System
Chip
© 2017-2019 Gyrfalcon Technology, Inc. 14© 2017-2019 Gyrfalcon Technology, Inc. 15
Thank You
© 2017-2019 Gyrfalcon Technology, Inc.CDVA Provides the Industry Standard…..
https://mpeg.chiariglione.org/sites/default/files/files/meetings/docs/w10117.do Compact descriptors for video
c
analysis for search and retrieval
applications:
1.Enable design of interoperable
object instance search applications
2.Ensure high matching performance
of objects
Approved Neural Network:
VGG16(Trained By ImageNet ILSVRC) –
No IP Issues
© 2017-2019 Gyrfalcon Technology, Inc. 17You can also read
