IEEE Santa Clara ComSoc/CAS Weekend Workshop Event-based analog sensing Theodore Yu theodore.yu@ti.com Texas Instruments Kilby Labs, Silicon Valley Labs September 29, 2012 1
Living in an analog world The world is analog Many different levels to sense Sight, sound, touch, taste, smell Analog interfaces are uniquely suited for each environment Increasingly, we turn to machines to help interpret the world for us Interface through sensors and actuators with computation being performed in digital machines e.g. microprocessors, cellphones, CPUs, etc. Digital computation is robust, easily configurable, and widespread 2
Analog-digital interface A D -Mostly digital Analog world is directly sampled into the digital domain e.g. all-digital implementations A D -Mostly analog Analog world is processed and interpreted in analog e.g. traditional analog implementations The placement of the boundary between analog and digital is flexible But transitions are expensive All-digital approach: send raw sensor data to digital domain Places the burden upon the analog-digital interconnect and digital processing power consumption All-analog approach: all-analog signal processing Often highly task specific which increases development time and reduces generalization to other applications 3
Analog-digital interface smart sensors A D -Mostly digital Analog world is directly sampled into the digital domain e.g. all-digital implementations A D -Mostly analog Analog world is processed and interpreted in analog e.g. traditional analog implementations The placement of the boundary between analog and digital is flexible But transitions are expensive Smart sensors and actuators Learning and interpretation of analog information Adaptation in analog sensor and actuator operation 4
Analog-digital interface Since the transition from analog domain to digital domain is expensive, only transmit what is necessary. Maximize information content of each digital bit Minimize transfer of redundant information Analog sensor interface Objective Operate analog circuits in high efficiency regime for low-power performance Integrated local analog signal processing circuitry results in sparse data being transferred to the digital domain Extract features of interest from sensors in the analog domain Transmit as digital events to the digital domain meaning? 0 1 0 0 1 0 Analog to digital encoding 5
Event-based sensing approach Each digital event encodes a feature of interest from the sensor Event encoding Feature selection Select what is and is not a feature from sensor data Decide what feature information to transmit for each event (i.e. spatial position, temporal position, etc.) Event decoding Digital processor must now interpret and understand what each event means Describes features of object as timebased digital events 0 1 0 0 1 0 Analog to digital encoding 6
Dynamic vision sensor (DVS) Frame-free image (scene) processing Only transmits individual pixel information when has a change in relative log intensity Characteristics Low bandwidth Low power consumption Low computational requirements High sensor dynamic range Technical specifications 128x128 resolution, 120dB dynamic range, 23mW power consumption, 2.1% contrast threshold mismatch, 15us latency http://www.youtube.com/embed/5nnoq1gq4sc Lichtsteiner, et. al. (ISSCC 2006, JSSC 2008)
A silicon retina that reproduces signals in the optic nerve Frame-free image (scene) processing Only transmits individual pixel information when has a change in relative log intensity Event decoding scheme ON activity corresponds to bright pixels and OFF activity corresponds to dark pixels Technical specifications <100mW power consumption, 3.5mm x 3.3 mm Zaghloul, et. al. (J. Neural Eng. 2006) 8
Convolution chips for image processing Event-based image processing Frame-free event-based image processing of asynchronous events On-the-fly processing of events results in 2-D filtered version of the input flow Characteristics Arbitrary kernel size and shape Technical specifications 32x32 pixel 2-D convolution event processor, 155ns event latency between output and input, 20Meps input rate, 45 Meps output rate, 350nm CMOS, 4.3x5.4mm 2, 200mW at maximum kernel size and maximum input event rate Linares-Barranco, et. al. (TCAS 2011)
Silicon cochlea architecture Input sound Seek to emulate cochlea performance and functionality by emulating cochlea biological architecture in silicon -2 nd order LPF bank -Transform into analog signal -Transform into digital neural event signal Chan, et. al. (TCAS I 2007) Digital events -Each event is a data packet describing event source (LPF) and event time
Reconstructed silicon cochlea data Input sound PC reconstructs the output digital event information by sorting by channel (LPF) number and then aligning according to time stamp information. Silicon cochlea Digital events channel number PC time
Example data with pure tones (for one channel) 300 Hz pure tone 750 Hz pure tone Simple real-time data processing procedure Count the time difference between events (interspike interval, ISI) for each channel channel number time channel number time Arrange the ISIs into a histogram A peak in the ISI histogram indicates a resonant frequency response bin count bin count ISI ISI
Sound Discrimination Example coo sound hiss sound Wav file FFT ISI histogram
3-D integrated silicon neuromorphic processor Park, et. al. (ISCAS 2012) Receiver Sender Hierarchical address-event routing (HiAER) Top metal 65,000, two-compartment neurons Conductance-based integrate and fire array transceiver (IFAT) 65 million, 32-bit virtual synapses Conductance-based dynamical synapses Dynamic table-look in embedded memory (2Gb DRAM) Locally dense, globally sparse synaptic interconnectivity Hierarchical address-event routing (HiAER) Dynamically reconfigurable Asynchronous spike event I/O interface 5 mm TSV HiAER IFAT (Analog CMOS) HiAER (Digital CMOS) DRAM 5 mm 0.13 m CMOS 5 mm IFAT Top metal I/O pad 0.13 m CMOS 5 mm 14
Event-driven framework Provide background on motivation Event-based approach relies upon temporal encoding to communicate signals. The time of the event is the key parameter, not the voltage value. Event-encoding is robust against additive noise. Coincidence detection performs efficient spikebased computation coincidence detection two or more arriving events result in a stronger response than a single arriving event applications event-driven sensing sensors are only on when something important happens event-driven computation information is sparsely represented with events Yu, et. al. (EMBC 2012) Theodore Yu UCSD Integrated Systems Neuroengineering Lab
Temporal code and synchrony At a local scale, neurons perform coincidence detection within temporal integration window. At a network scale, the temporal delay information in events models the spatial distribution between neurons. Each scene of interest can be encoded as a unique combination of features Input pattern 10ms delay 5ms delay Coincidence? Yes or no? 4ms delay 16
Temporal code and synchrony example Event at t = 7ms Event at t = 3ms 10ms delay 5ms delay Coincidence? No! Event at t = 8ms 4ms delay Event at t = 7ms Event at t = 2ms 10ms delay 5ms delay Coincidence? Yes! Event at t = 8ms 4ms delay 17
Summary Analog event-based sensing Since the transition from analog domain to digital domain is expensive, only transmit what is necessary. Maximize information content of each digital event through encoding of features in analog domain Minimize transfer of redundant information for sparse digital signal processing Applications Visual and acoustic sensors for event-encoding of features Event-based processor performs event-decoding of features utilizing coincidence detection in neural synchrony 18
Thank you 19