Optogenetic visual cortical prosthesis - Dr. Patrick Degenaar Conflicts of interest: I have filed a number of patents and CANDO may have commercial
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
OptoNeuro
Optogenetic visual cortical prosthesis
Conflicts of interest: I have filed a number of patents and CANDO may have commercial
impact. But currently no commercial interests.
Dr. Patrick Degenaar
Reader in Neuroprosthesis
Neuroprosthetics @ Newcastle
Epilepsy brain prosthesis
http://www.cando.ac.uk
Visual prosthesis
http://www.optoneuro.eu
Spinal cord injury Stimulate Record
Hand/Leg bionics
Electrodiagnostics
Process
Multi-EMG recording
Closed loop neural
interfaces!!
The optogenetics revolution
Blue light is used to activate neurons expressing
channelrhodopsin. Other opsins can silence activity
Optogenetic control of behaviour.
Gradinaru et al. J Neurosci (2007)
Optoelectronics for optogenetics
µlens optics
Gallium
Nitride µLED GaN MQW
3D bump bond p-contact
Solder bond
CMOS driver CMOS
64x64, 64x1 16 x 16 MPW CMOS 90x90 Retina CANDO 1 CANDO 3
passive µLED arrays chip driven µLED arrays wafer Optrode Wafer
2009 2010 2011 2012 2013 2014 2015 2016 2017 2018
High radiance optoelectronics for optogenetics:
Grossman N et al. IEEE TBME 2011, 58(6), 1742-1751.
McGovern B et al IEEE TBCAS 2010, 4(6, part 2), 469-476.
Grossman N et al, J. Neural Engineering 2010, 7(1), 016004.
The visual system
Optics
(cornea/lens)
Temporal
Sense/code
(retina)
Transmission Pulvinar nucleus
(Optic Nerve) Lateral Geniculate
Nucleus
Sorting Superior
Colliculus
(Thalamus)
Optical
radiation
Interpretation Primary visual
(Visual Cortex) cortex
5
Visual loss: statistics
Corneal opacities (5%)
Diabetic retinopathy (5%)
According to WHO, about Prevalence Childhood blindness (4%)
180 million people worldwide
have a visual impairment AMD
(9%)
Others Glaucoma
Legal blindness: (16%) (12%)
“medically diagnosed central
visual acuity of 20/200 or less
in the better eye with the best Cataracts
possible correction, and/or a (48%)
RP (1%)
visual field of 20 degrees or Trauma
less” (?%)
From: WHO Primary visual
prosthesis target
conditions
Drugs and gene therapy becoming increasingly
useful.. But cannot restore lost vision
6
Disorders of the visual system
Optics
(cornea/lens) Sub/epi retinal
prosthesis
Sense/code Retinitis Pigmentosa
Optic nerve
(retina)
prosthesis (
Implementing a visual prosthesis
Optics
(cornea/lens)
Image
Sense/code acquisition
(retina)
Transmission
(Optic Nerve) Visual
processing
Sorting
(Thalamus) Wireless
power/data
Stimulator
Interpretation
(Visual Cortex)
Easy??
Brindley and Lewin did it in
1968!!!
• Brindley,G.S.,Lewin,W.S.,1968.The visual sensations produced by electrical stimulation of the medial
occipital cortex. J Physiol.19454-5P.
• Brindley,G.S.,Lewin,W.S.,1968.The sensations produced by electrical stimulation of the visual
cortex.J.Physiol.196,479–493.
Challenge 1: Sheer scale of challenge
1960’s: You could build a device on a Monday
and put it in a patient on a Friday
Now: A project to create an active implantable
device requires £10-50M and 5-10 years
worth of regulatory effort to demonstrate
safety before implantation is allowed
As such, almost the entire field moved to retinal
prosthesis in the 1990’s as it was a much easier target!!Challenge 2: Visual acuity
Asian/pictographic
scripts:
~ 1000 pixels
European/phonetic
scripts:
~ 5000 pixels
100
105 xx 100
33
70 10
33
70
5Normal
(10,000)
(100)
(1000)
(5000)
(25) vision pixels European/phonetic
characters:
~ 100 pixels
11Lessons from retinal prosthetics
1500 electrodes
– but only a few
dozen can be on
at any moment in
time.
From Retina AGChallenge 3: Effective contrast
Assume greyscale retina Add noisy retina
Original image @ 64 x 64 resolution effect
With insufficient
stimulus contrast it
is like whispering
through a
typhoon!
100:1 contrast ratio 50:1 contrast ratio
13Challenge 4: The eye is not a camera!
The retina
Rods/cones
1. Image capture
Ligtht
Horizontal cells
2. smoothing
5. Spike coding Bipolar cells
Ganglion cells 3. Derivative
OFF ON
Amacrine cells
4. Temporal edge
The retina has 60 different cell types. If we are bypassing the
natural processing, we need to replicate the key features!
14Challenge 5: Biocompatibility
Glial scarring
Electrode
insertion
point
Journal of The Electrochemical
Society, 152 (7) J85-J92 ~2005! Accelerated ageing, 1000 IEEE Transactions on Neural Systems and
Rehabilitation , Vol. 12, No. 2, June 2004
cycles at 1.7V/s
Ideal Real
Electrode
degradation and
adverse tissue
reactions are a major
issue for smaller
stimulating
electrodes because
of high charge
density requirements
Annu. Rev. Biomed. Eng.
10:275–309, 2008 15Where to start?
Developing the stimulator
Surface stimulation
(Brindley, Dobelle)
0.18 mm
0.18 mm
0.46 mm
0.41 mm 2.2 mm
0.31 mm
`
0.68 mm
From: Atlas of Cytoarchitectonics
- Von Economo
Penetrating stimulation
Can we make the communication bi-directional?
Can we treat communication as a control problem?Optogenetics & closed loop control
Optical 0.18 mm
0.18 mm
Electricall 0.46 mm
Light stimulus
0.41 mm 2.2 mm
Electrical recording
0.31 mm
0.68 mm
From: Atlas of Cytoarchitectonics
- Von Economo
Penetrating stimulation
Can we make the communication bi-directional?
Can we treat communication as a control problem?Controlling Abnormal Network Dynamics with Optogenetics
£10M project: Clinical trials aimed for 2021
19Noise cancellation of epileptic
seizures – but adaptable to
visionAdapting to visual brain prosthetics
Brain Brain implant unit (Actually
implant unit different position)
Control unit
Connecting
lead
Central control unit
inc wireless comms
External
headset
CANDO - Epilepsy Visual brain prosthesis
Principles: 2mm 1mm 4mm
1. Custom brain implant starting as 4 x
4 x 4 LEDs can be utilised TX Skin Fat Muscle RX
2. The control unit is quite different –
higher power, no battery Average 5 mm
3. Cabling can be simplerBrain stimulator unit
Communications Power
Digital control unit (FSM)
Recording Diagnostic Stimulation
Circuits Circuits Circuits
Interface unit Optrode head
Recording
Active electronics Electrode
Optrode Mounting Photonic
plate Stimulation
site
1. Ramezani et al. submitted to IEEE TCAS-I, June 2017
Optrode
2. Zhao et al. Implantable optrode GB1410886.4
shaft
3. Dekhoda et al. Temperature sensor GB1522790.3
4. Degenaar Optical stimulation arrangement GB1616725.6
5. Constandinou et al On Chip Random ID generation GB1702623.8Recording microelectronics
C
Vin
V VON
Vref 50C LNA VIP VIN OP 6C Gm VIP VIN
VOUT
C=100fF
C
Front end amplifier Programmable gain amplifier
Recordings from non-human primate brainLED driver circutry
• Amplitude modulation
• PWM modulation
• Diagnostic and methods
LED drive power LED efficiency
DACHow much light do we need?
Lambertian emitter Shaped emitter Pseudo-collimated emitter
Threshold = 1mW/mm2 Threshold = 1mW/mm2 Threshold = 1mW/mm2
100
Optical Power (mW)
10
1
0.1
(a) 0.01
(d) (e) (f)
0.001
0 200 400 600 800 1000 0 200 400 600 800 1000 0 200 400 600 800 1000
Tissue penetration depth (μm)
LED diameter
120μm 100μm 80μm 60μm 40μm
Threshold = 1mW/mm2 Threshold = 0.1mW/mm2 Threshold = 0.01mW/mm2
100
Key take home message:
101. Size doesn’t matter to penetration depth!
Optical Power (mW)
(b)
12. Need to penetrate at least 100-200µm for long term implantables due to gliosis
0.1
3. More collimated light will give better penetration
0.01
0.001
Dong et al. For submission to J. Biomedical Photonics 20170.01 0.01
Optical stimulus Optical stimulus
LED thermal impact
(a) 1ms 10ms 100ms 1s 10s (b) 1ms 10ms 100ms 1s 10s
0 1.0 2.0 3.0 Dimension (μm)
ΔT (°C)
ΔT (K)
2.0
ΔT (°C)
3P 10
ΔT(K)
0 .0 0 0
0 .1 0 0 0
200
2.0 0 .2 0 0 0
(e) II
1P LED on 10mW 8 LEDs on 10mW
0 .3 0 0 0
1.5 0 .4 0 0 0 Optrode 1.5
L
200
0 .5 0 0 0
1.0 0 .6 0 0 0 profile
0 .7 0 0 0
ΔT (K) II 0.5
8
0 .8 0 0 0
ΔT(K)
Dimension (μm)
0 .9 0 0 0
1.0
0 .0 0 0
0 .1 0 0 0 S 0 1 .0 0 0
S 1.0
0.8
0 .2 0 0 0
0 .3 0 0 0
III
2
0.6
0 .4 0 0 0
0 .5 0 0 0
0.5
6
0.4 0 .6 0 0 0
0 .7 0 0 0
0.2 0 .8 0 0 0
I (d) I
5mW
0 .9 0 0 0
0 1 .0 0 0
100μm (b)
5mW
0
2.0
ΔT (K)
4
0 .0 0 0
2.0
0 .1 0 0 0
0 .2 0 0 0
(f) III
L
1
0 .3 0 0 0
P 1.5 0 .4 0 0 0
1.5
E
2.5mW 2.5mW
0 .5 0 0 0
1.0
ΔT(K)
0 .6 0 0 0
0.000
2
0 .7 0 0 0
0.1000
0.5 0 .8 0 0 0
L
0 .9 0 0 0
0 1 .0 0 0
1.0
E
0.2000
0.3000
S
0 0.5
0
0.4000
0.5000
0.6000
0 Optical stimulus Optical stimulus
1000μm
0.7000
Dimension (μm) 1000
0.8000
No coating 20um polymer coating
1ms
100μm 10ms 100ms 1s 10s 1ms 10ms 100ms 1s 10s
0.9000
100μm (a) (c) 1.000 On-optrode On-optrode
L: LED E: Electrode
S: Silicon substrate
(c)
P: Polymer coating 25um in the tissue
100um in the tissue (d)
Maximum allowed pulse width Maximum allowed duty ratio (%)
10s 100
Target LED characteristics: Typical micro-LED efficiencies from
literature
80 ~ 1-5%
1s
Driving current: 1mA
Driving voltage: 4-5V Wu et
60 al. Neuron 2015: 0.8%
0.1s
1 LED on Kim et al. Science 2013: 0.2%
1 LED on
Light requirement: 1mW McAlinden
40 al: Opt Lett 2013: 5%
10ms McGovern et al: PLOS ONE 2013: 5%
Efficiency: 8 LEDs on>20% 8 LEDs
McGovern et al: IEEEonTBCAS 2010:1%
20
1ms
0
20 30 40 50 60 10 0 10 20 30 40 50 60
(e)Dong et al. For
Thermal to J. Biomedical(f)Photonics 2017
power (mW)
submission Thermal power (mW)LED efficiencies
Wu et al Neuron 2015
Our LED 100µm 30µm 10µm @ 1mA
320 x 240!
Dong et al. For submission to J. Biomedical Photonics 2017Light requirement in-vivo
“Official” threshold is
0.7mW/mm2 for dissociated
culture. But what about in
practice???
In NHP experiments, We saw
responses at 1mm with emission
intensities of only 0.5mW. i.e. At
that depth the irradianceAssembling the optrodes
Schematic design CAD layout Wafer fabrication
Probe assembly 3D assembly
Control
electronics
b)
Light-
emitting
diodes
Recording
sites
We are developing a fully manufacturable implant!!Developing the control system
Ontogenetically
Sensitized tissue
Optrode
Implant
Base plate
Internal
control unit
Camera Receiver
Transmitter Power and antennae To receiver
antennae Control unit antennae
Batt and cam External control unit Internal control unit Optrode unit
Power Power Power RX/ Voltage
3.7V TX RX
module trans/Amp AC->DC regulator
USB Embedded BLE BLE Embedded
SoC module module SoC
Optrodes
Fattah & Soltan et al. in preparation for IEEE TBME submissionSystem software - preprocessing
Visual augmentation
The vision that we will
return to patients with
visual prosthesis will be
rudimentary. We are
therefore using patients
with partially degenerate
vision as a simulation of
Cartoonization what can be achieved.
Simplification
Song of the Machine!Full software process
Ontogenetically
Sensitized tissue
Optrode
Implant
Base plate
Internal
control unit
Camera Receiver
Transmitter Power and antennae To receiver
antennae Control unit antennae
External control unit Internal control unit
Compression
Video/Image
acquisition
Decompression Pulse coding
Transmission
Simplification Retinal processing LED reconstruction
Fattah & Soltan et al. in preparation for IEEE TBME submissionVisual cortical Headset
Implantable control unit
44% 61% 75% 86%
80 80
70 70
MC Perfromance (fps)
60
60
Power (mW)
50
40 50
30 40
20
30
10
0 20
60 40 20 60 40 20
Frequency (MHz) Frequency (MHz)Timeline
Visual cortical trial??
CANDO trial
CANDO Engineering
CANDO First
in human trial
CANDO Current
Regulatory & funding ends
We are here ethical (Aug 2021)
submissionAcknowledgments
OptoNeuro
CANDO project
Tim Constandinou (Imperial College), Nick Donaldson
(UCL), Andrew Sims (Newcastle RVI (NHS)), Andrew
Jackson (Newcastle IoN), Mark Cunningham
(Newcastle IoN), Anthony O’Neil (Newcastle EEE),
+ Others:
OptoNeuro FP7 project
Mark Neil (Imperial College), Ernst Bamberg (Max
Planck Institute), Christian Bamann (Max Planck
Institute), Botond Roska (FMI, Switzerland), Pleun
Maaskant (Tyndall Institute), David Rogerson
(Scientifica), Mark Johnson (Scientifica)You can also read
