AN ANALOG VLSI SYSTEM FOR STEREOSCOPIC VISION

AN ANALOG VLSI SYSTEM FOR STEREOSCOPIC VISION

THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE VLSI, COMPUTER ARCHITECTURE AND DIGITAL SIGNAL PROCESSING Consulting Editor Jonathan Allen Other books in the series: ANALOG DEVICE-LEVEL LAYOUT AUTOMATION, John M. Cohn, David J. Garrod, Rob A. Rutenbar, L. Richard Carley ISBN: 0-7923-9431-3 VLSI DESIGN METHODOLOGIES FOR DIGITAL SIGNAL PROCESSING ARCHITECTURES, Magdy A. Bayoumi ISBN: 0-7923-9428-3 CIRCUIT SYNTHESIS WITH VHDL, Roland Airiau, Jean-Michel Berge, Vincent Olive ISBN: 0-7923-9429-1 ASYMPTOTIC WAVEFORM EVALUATION, Eli Chiprout, Michel S. Nakhla ISBN: 0-7923-9413-5 WAVE PIPELINING: THEORY AND CMOS IMPLEMENTATION, C. Thomas Gray, Wentai Liu, Ralph K. Cavin, III ISBN: 0-7923-9398-8 CONNECTIONIST SPEECH RECOGNITION: A Hybrid Appoach, H. Bourlard, N. Morgan ISBN: 0-7923-9396-1 BiCMOS TECHNOLOGY AND APPLICATIONS, SECOND EDITION, A.R. Alvarez ISBN: 0-7923-9384-8 TECHNOLOGY CAD-COMPUTER SIMULATION OF IC PROCESSES AND DEVICES, R. Dutton, Z. Yu ISBN: 0-7923-9379 VHDL '92, THE NEW FEATURES OF THE VHDL HARDWARE DESCRIPTION LANGUAGE, J. Berge, A. Fonkoua, S. Maginot, J. Rouillard ISBN: 0-7923-9356-2 APPLICATION DRIVEN SYNTHESIS, F. Catthoor, L. Svenson ISBN :0-7923-9355-4 ALGORITHMS FOR SYNTHESIS AND TESTING OF ASYNCHRONOUS CIRCUITS, L. Lavagno, A. Sangiovanni-Vincentelli ISBN: 0-7923-9364-3 HOT-CARRIER RELIABILITY OF MOS VLSI CIRCUITS, Y. Leblebici, S. Kang ISBN: 0-7923-9352-X MOTION ANALYSIS AND IMAGE SEQUENCE PROCESSING, M. I. Sezan, R. Lagendijk ISBN: 0-7923-9329-5 HIGH-LEVEL SYNTHESIS FOR REAL-TIME DIGITAL SIGNAL PROCESSING: The Cathedral-II Silicon Compiler, J. Vanhoof, K. van Rompaey, I. Bolsens, G. Gossens, H. DeMan ISBN: 0-7923-9313-9 SIGMA DELTA MODULATORS: Nonlinear Decoding Algorithms and Stability Analysis, S. Hein, A. Zakhor ISBN: 0-7923-9309-0 LOGIC SYNTHESIS AND OPTIMIZATION, T. Sasao ISBN: 0-7923-9308-2 ACOUSTICAL AND ENVIRONMENTAL ROBUSTNESS IN AUTOMATIC SPEECH RECOGNITION, A. Acero ISBN: 0-7923-9284-1 DESIGN AUTOMATION FOR TIMING-DRIVEN LAYOUT SYNTHESIS, S. S. Sapatnekar, S. Kang ISBN: 0-7923-9281-7

AN ANALOG VLSI SYSTEM FOR STEREOSCOPIC VISION by Misha Mahowald M.R.e. Anatomical Neuropharmacology Unit Oxford, England... " SPRINGER-SCIENCE+BUSINESS MEDIA, LLC

Library of Congress Cataloging-in-Publication Data Mahowald, Misha. An analog VLSI system for stereoscopic vision / Misha Mahowald. p. cm. -- (The Kluwer international series in engineering and computer science. VLSI, computer architecture, and digital signal processing) Includes bibliographical references and index. ISBN 978-1-4613-6174-9 ISBN 978-1-4615-2724-4 (ebook) DOI 10.1007/978-1-4615-2724-4 1. Computer vision. 2. Stereoscopic views. circuits. 1. Title. II. Series. TA1634.M35 1994 006.3 '7 --dc20 3. Linear integrated 93-49498 CIP Copyright 1994 by Springer Science+Business Media New York OriginalIy published by Kluwer Academic Publishers in 1994 Softcover reprint of the hardcover 1 st edition 1994 AlI rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise, without the prior written permission of the publisher, Springer-Science+Business Media, LLC. Printed an acid1ree pa per.

CONTENTS FOREWORD PREFACE ACKNOWLEDGMENTS IX Xl Xlll 1 SYNTHESIS 1 2 THE SILICON RETINA 4 2.1 Anatomical Models 4 2.1.1 Feedforward Model: Logarithms. Differences. and Perceptual Invariance 8 2.1.2 Recurrent Model: Amplification and Adaptation 10 2.2 Architecture of the Silicon Retina 14 2.3 Photoreceptors 2.3.1 Computation of the Logarithm 2.3.2 Shift of the Logarithm 2.4 Horizontal Cells 2.4.1 Passive Resistive Network 2.4.2 Active Resistive Network 2.5 Bipolar Cells 2.5.1 Time Response 2.5.2 Edge Response 2.6 Physical Constraints on Information Processing 2.6.1 Bandwidth 2.6.2 Parametric Control 2.6.3 Wiring Density 19 21 24 31 34 39 43 43 47 52 52 54 54 v

vi AN ANALOG VLSI SYSTEM FOR STEREOSCOPIC VISION 2.7 EfllergentPToperties 56 2.7.1 Afterifllages 56 2.7.2 Sifllultaneous Contrast 59 2.7.3 Mach Bands 61 2.7.4 Heffllann--Hering Grid Illusion 61 3 THE SILICON OPTIC NERVE 66 3.1 SUfllfllary of Existing Techniques 66 3.2 Address-Event Representation 68 3.3 Model of Data-Transfer Tiflling Efficiency 71 3.4 Data Transfer in One Difllension 74 3.4.1 Action Potential 74 3.4.2 One-Difllensional Arrays 80 3.4.3 Arbiter 87 3.5 Two-Difllensional Retina--Receiver SyStefll 91 3.5.1 Receiver 91 3.5.2 Sender Retina 93 3.5.3 Ifllage Data 96 3.6 Advantages of Address Events 107 3.6.1 The Future: Multichip SYStefllS 107 3.6.2 Flexibility of Digital Abstraction 112 3.6.3 Action Potentials in Biological SYStefllS 113 4 STEREOPSIS 118 4.1 Stereocorrespondence 121 4.2 Neurophysiology 125 4.3 Stereocorrespondence AlgOrithfllS 125 4.3.1 Cooperative AlgOrithfllS 127 4.3.2 Multiresolution AlgOrithfllS 128 4.3.3 Cooperative Multiresolution AlgOrithfll 132 4.4 Stereocorrespondence Chip 138 4.4.1 Correlators 139 4.4.2 Winner-Take-All Inhibition 144 4.4.3 Analog Disparity Units 148 4.4.4 Control of Positive Feedback 152 4.4.5 Monocular Units 155

Contents vii 4.5 Experiments 159 4.5.1 Methods 159 4.5.2 Tilted Surfaces 160 4.5.3 Parameters 164 4.5.4 Occlusion 170 4.5.5 Disparity-Gradient Limit 174 4.6 Stereocorrespondence as a Model of Cortical Function 177 5 SYSTEM 182 A SIMPLE CIRCUITS 192 A.1 Transistors 192 A2 Current Mirrors 193 A3 Differential Pairs 196 A4 Transconductance Amplifiers 196 A5 Low-Pass Filter 198 A6 Resistor 198 BIBLIOGRAPHY 203 INDEX 213

FOREWORD It is seldom that the essence of an entire field is represented in a single small volume. Dr. Mahowald has managed to span the space from neurons to microcircuits, and from photoreceptors to cortical processing of visual space. Her treatment of stereopsis is detailed and compelling, reaching well beyond algorithmic models to grapple with the nature of information representation in the physical medium---neurons or silicon. The interplay of algorithm and information representation is well known in the realm of traditional computer programs. That interplay is, however, one of the least understood aspects of neural computation. This aspect of the nervous system cannot be approached with observation alone---it requires the construction of credible models that can be compared directly with neurobiological experiment. Mahowald succeeds brilliantly in divining a role for each of the four known types of stereo-tuned cells in her analog model that operates in real time on real-world images. In the process she makes a convincing case that neural circuits can be better modeled by electronic circuits than by conventional sequential computer languages. This classic monograph is required reading for anyone with serious interest neural computation, and will be of value to anyone interested in neurobiology or large-scale analog circuits. Carver Mead ix

PREFACE This text is an extensively modified version of my doctoral dissertation, submitted to the Department of Computation and Neural Systems at the California Institute of Technology in 1992. I studied there under the guidance of Carver Mead, who pioneered the field of neuromorphic analog VLSI design. His book, Analog VLSI and Neural Systems (Addison-Wesley, 1989), lays out the principles and basic circuits on which this work is based. This text is a cinema verite description of VLSI analogs of neuronal visual processing. In this work, my goal is to investigate the interaction of the physical medium and the computation in both biological and analog VLSI systems by synthesizing a functional neuromorphic system in silicon. In both the synthesis and analysis of the system, I adopt the point of view of the machine---a point of view from within the system---rather than that of an omniscient designer drawing a blueprint. This perspective projects the design and the designer into a living landscape. The motivation for a machine-centered perspective is explained in the first chapter. The second chapter describes the evolution of the silicon retina. The retina accurately encodes visual information over orders of magnitude of ambient illumination, using mismatched components that are calibrated as part of the encoding process. The visual abstraction created by the retina is suitable for transmission through a limited bandwidth channel. The third chapter introduces a general method for interchip communication, the address-event representation, which is used for transmission of retinal data. The address-event representation takes advantage of the speed of CMOS relative to biological neurons to preserve the information of biological action potentials using digital circuitry in place ofaxons. The fourth chapter describes a collective circuit that computes stereodisparity. In this circuit, the processing that corrects for imperfections in the hardware compensates for inherent ambiguity in the environment. The fifth chapter demonstrates a primitive working stereovision system. This work has contributed to both computer engineering and neuroscience at a concrete level. Through the construction of a working analog of biological vision subsystems, I have developed new circuits for building brainxi

xii AN ANALOG VLSI SYSTEM FOR STEREOSCOPIC VISION style analog computers, and have explained specific neurophysiological and psychophysical results in terms of underlying electronic mechanisms. These examples demonstrate the utility of using biological principles for building brain-style computers and the significance of building brain-style computers for understanding the nervous system.

ACKNOWLEDGMENTS The preparation of this manuscript has taken longer than anticipated. To people who put up with my preoccupation with Latex, when I should have been studying the cortex---john Anderson, Charmaine Nelson, the Medical Research Council Anatomical Neuropharmacology Unit in Oxford, and the Office of Naval Research in the United States---thank you for supporting me during this project. I am especially grateful to the people in Oxford who read many provisional words and suggested better ones: Bashir Ahmed, Rodney Douglas, and Kevan Martin. Thanks also to Pam Twynam for unflagging assistance with bibliographic formatting. The roots of this work reach halfway around the world, to the fertile environment of Carver Mead's laboratory at the California Institute of Technology. In my doctoral dissertation, I acknowledged those people who participated in that phase of the work; here, I would like to acknowledge those people who contributed further to this particular manuscript: Helen Derevan and Donna Fox, who have been my hands and ears in California; Lyn Dupre, who began the revision of my dissertation with a thorough red pen; Tobi Delbruck, who has kept me informed of ongoing retina projects; Kwabena (Buster) Boahen and John Lazzaro, who have clarified the address-event protocol. Thank you all. I thank the members of my examining committee: Jerry Pine, John Allman, Pietro Perona, Alain Martain, and Carver Mead; as well as Francis Clauser and his family, whose enthusiastic support encouraged me to undertake this revision. I repeat the heartfelt mantra of researchers in VLSI design: thanks be to MOSIS for providing fabrication and to the funding agency that supports us, ARPA. xiii

AN ANALOG VLSI SYSTEM FOR STEREOSCOPIC VISION