Real-time Processing of Multiplexed Data Acquired via Flexible Active Electrode Arrays

Similar documents
Portable in vivo Recording System

ECE 4220 Real Time Embedded Systems Final Project Spectrum Analyzer

LabView Exercises: Part II

International Journal of Engineering Research-Online A Peer Reviewed International Journal

Major Differences Between the DT9847 Series Modules

ni.com Digital Signal Processing for Every Application

VGA Controller. Leif Andersen, Daniel Blakemore, Jon Parker University of Utah December 19, VGA Controller Components

Lab experience 1: Introduction to LabView

PulseCounter Neutron & Gamma Spectrometry Software Manual

Data Converters and DSPs Getting Closer to Sensors

Brain-Computer Interface (BCI)

TECHNICAL SPECIFICATIONS, VALIDATION, AND RESEARCH USE CONTENTS:

PRELIMINARY INFORMATION. Professional Signal Generation and Monitoring Options for RIFEforLIFE Research Equipment

MSO-28 Oscilloscope, Logic Analyzer, Spectrum Analyzer

IEEE Santa Clara ComSoc/CAS Weekend Workshop Event-based analog sensing

Figure 1: Feature Vector Sequence Generator block diagram.

Exercise 1-2. Digital Trunk Interface EXERCISE OBJECTIVE

An Introduction to the Spectral Dynamics Rotating Machinery Analysis (RMA) package For PUMA and COUGAR

VivoSense. User Manual Galvanic Skin Response (GSR) Analysis Module. VivoSense, Inc. Newport Beach, CA, USA Tel. (858) , Fax.

Getting Started with the LabVIEW Sound and Vibration Toolkit

Multiwell-MEA-System

MindMouse. This project is written in C++ and uses the following Libraries: LibSvm, kissfft, BOOST File System, and Emotiv Research Edition SDK.

MIE 402: WORKSHOP ON DATA ACQUISITION AND SIGNAL PROCESSING Spring 2003

Pre-processing of revolution speed data in ArtemiS SUITE 1

Spectrum Analyser Basics

Elasticity Imaging with Ultrasound JEE 4980 Final Report. George Michaels and Mary Watts

Multi-Parameter Monitoring Data Acquisition System for SpO 2 Signals

Real-time EEG signal processing based on TI s TMS320C6713 DSK

CAEN Tools for Discovery

Introduction To LabVIEW and the DSP Board

Introduction: Overview. EECE 2510 Circuits and Signals: Biomedical Applications. ECG Circuit 2 Analog Filtering and A/D Conversion

A Matlab toolbox for. Characterisation Of Recorded Underwater Sound (CHORUS) USER S GUIDE

Digital Effects Pedal Description Ross Jongeward 10 December 2014

Environmental Controls Laboratory

Digital Logic. ECE 206, Fall 2001: Lab 1. Learning Objectives. The Logic Simulator

New Generation of MEA-Systems: MEA2100-System

The Measurement Tools and What They Do

Design Project: Designing a Viterbi Decoder (PART I)

A dedicated data acquisition system for ion velocity measurements of laser produced plasmas

Integration of Virtual Instrumentation into a Compressed Electricity and Electronic Curriculum

REPORT DOCUMENTATION PAGE

B I O E N / Biological Signals & Data Acquisition

1ms Column Parallel Vision System and It's Application of High Speed Target Tracking

Chapter 1. Introduction to Digital Signal Processing

Distortion Analysis Of Tamil Language Characters Recognition

Digital Video Telemetry System

Low Power VLSI Circuits and Systems Prof. Ajit Pal Department of Computer Science and Engineering Indian Institute of Technology, Kharagpur

Experiment 13 Sampling and reconstruction

DT9857E. Key Features: Dynamic Signal Analyzer for Sound and Vibration Analysis Expandable to 64 Channels

CHAPTER 3 ECG SIGNAL RECORDING USING LABVIEW

Lab 1 Introduction to the Software Development Environment and Signal Sampling

A Real Time Infrared Imaging System Based on DSP & FPGA

Yong Cao, Debprakash Patnaik, Sean Ponce, Jeremy Archuleta, Patrick Butler, Wu-chun Feng, and Naren Ramakrishnan

Installation of a DAQ System in Hall C

Reconfigurable Neural Net Chip with 32K Connections

1.1 Digital Signal Processing Hands-on Lab Courses

Scan. This is a sample of the first 15 pages of the Scan chapter.

Meeting Embedded Design Challenges with Mixed Signal Oscilloscopes

TV Character Generator

Power Consumption Trends in Digital TVs produced since 2003

Speech and Speaker Recognition for the Command of an Industrial Robot

Microbolometer based infrared cameras PYROVIEW with Fast Ethernet interface

UNIVERSAL SPATIAL UP-SCALER WITH NONLINEAR EDGE ENHANCEMENT

AN ARTISTIC TECHNIQUE FOR AUDIO-TO-VIDEO TRANSLATION ON A MUSIC PERCEPTION STUDY

USING MATLAB CODE FOR RADAR SIGNAL PROCESSING. EEC 134B Winter 2016 Amanda Williams Team Hertz

ME EN 363 ELEMENTARY INSTRUMENTATION Lab: Basic Lab Instruments and Data Acquisition

Multiband Noise Reduction Component for PurePath Studio Portable Audio Devices

Blending in action: Diagrams reveal conceptual integration in routine activity

Heart Rate Variability Preparing Data for Analysis Using AcqKnowledge

PICOSECOND TIMING USING FAST ANALOG SAMPLING

DELTA MODULATION AND DPCM CODING OF COLOR SIGNALS

DT3162. Ideal Applications Machine Vision Medical Imaging/Diagnostics Scientific Imaging

Experiment 2: Sampling and Quantization

A Parametric Autoregressive Model for the Extraction of Electric Network Frequency Fluctuations in Audio Forensic Authentication

Digital Strobe Tuner. w/ On stage Display

Exercise 2-1. External Call Answering and Termination EXERCISE OBJECTIVE

Choosing an Oscilloscope

Image Acquisition Technology

technical note flicker measurement display & lighting measurement

A BCI Control System for TV Channels Selection

In the development of electronic instruments or the analysis

DT8837. High Performance Ethernet Instrument Module for Sound & Vibration. Overview. Key Features

CHARACTERIZATION OF END-TO-END DELAYS IN HEAD-MOUNTED DISPLAY SYSTEMS

ECE438 - Laboratory 4: Sampling and Reconstruction of Continuous-Time Signals

Realizing Waveform Characteristics up to a Digitizer s Full Bandwidth Increasing the effective sampling rate when measuring repetitive signals

A. Chatterjee, Georgia Tech

EXPLORING THE USE OF ENF FOR MULTIMEDIA SYNCHRONIZATION

Part 1: Introduction to Computer Graphics

HBI Database. Version 2 (User Manual)

Objectives. Combinational logics Sequential logics Finite state machine Arithmetic circuits Datapath

Work In Progress: Adapting Inexpensive Game Technology to Teach Principles of Neural Interface Technology and Device Control

EEG Eye-Blinking Artefacts Power Spectrum Analysis

Comp 410/510. Computer Graphics Spring Introduction to Graphics Systems

A New "Duration-Adapted TR" Waveform Capture Method Eliminates Severe Limitations

2. AN INTROSPECTION OF THE MORPHING PROCESS

Re: ENSC 370 Project Physiological Signal Data Logger Functional Specifications

Broadcast Television Measurements

SDR Implementation of Convolutional Encoder and Viterbi Decoder

Agilent MSO and CEBus PL Communications Testing Application Note 1352

Portable USB Potentiostat Low-Current Portable USB Potentiostat Extended Voltage USB Potentiostat

Transcription:

Real-time Processing of Multiplexed Data Acquired via Flexible Active Electrode Arrays Dept. of CIS - Senior Design 2010-2011 Robert B. Yaffe yaffer@seas.upenn.edu Univ. of Pennsylvania Philadelphia, PA Daniel S. Rosenthal danr@seas.upenn.edu Univ. of Pennsylvania Philadelphia, PA Brian Litt littb@mail.med.upenn.edu Univ. of Pennsylvania Philadelphia, PA ABSTRACT A new generation of flexible active electronics have been developed to record from biological tissue. These electrodes conform to biological surfaces, such as the brain, and can provide recordings of high spatial resolution. However, the previous data acquisition system for these electrodes suffered from limitations regarding the collection and storage of data. Multiplexed data was recorded from the electrodes and then it was saved. It possessed a limited display which provided little feedback to researchers during experiments. The data was later converted and loaded into MATLAB (The MathWorks, Natick, MA) and processing was done offline. A real-time online processing system has been built for the new generation of flexible active electronics. Raw voltage inputs are demultiplexed and filtered in real-time. Evoked response averages are generated for each electrode and a GUI (graphical user interface) is generated that displays a color map of real-time features. A 2D (2-dimensional) sparse noise plot is implemented. Demultiplexed data is saved to disk to allow for offline analysis. Experimenters can now use provided displays to analyze the results of experimentation in real-time and to adjust experimental parameters accordingly. The system has been tested and is being prepared to be used in future experiments. 1. INTRODUCTION While recordings from the human brain can provide rich and informative data sets, current data acquisition methods make quantitative analysis a computationally intensive process. Electrodes, which can be placed above or below the scalp, give crude depictions of the neural networks underneath. Furthermore, because sampling rates need to be high for research purposes, files generated by acquisition systems are massive (on the order of terabytes) and often require extensive processing. Understanding brain recordings thus becomes a task of interpretation and computation. Putting technical limitations aside, developing a firm understanding of these complex biological networks is important to further clinical research. The information provided by a simple array of electrodes can be represented in a number of ways. By isolating neuronal activity after a presented stimulus, evoked responses can be extracted and averaged to show the response of various brain regions to the stimulus. From these evoked responses, color maps can be drawn to give spatial exposition of the most active regions of the underlying tissue. Fourier analysis can be performed to provide frequency content of the signal and power spectra. While the list continues, data significance is still limited by the crude representation of the actual neural activity due to physical limitations of the recording electrodes. To combat the problem of low spatial resolution, Kim and Viventi et al [2] are developing new flexible subdural electrode arrays that conform to biological tissue. The biointerfaced system contains a substrate of silk fibroin which adheres to the curvilinear surface of the brain and gives a much clearer picture of neural activity than previously developed hard, planar electrodes. To read the underlying neurons, the acquisition system works by having multiple electrodes feed data through common channels. By transmitting multiple analog signals through a shared medium, a process known as multiplexing, researchers minimize the amount of implantable hardware required. At a given point in time, only a single row of electrodes feeds voltages to the acquisition software. By rapidly changing the row selected, the signal can be transmitted with significantly fewer wires. Furthermore, this multiplexing requires that the electrodes be sampled at a much higher rate to ensure that aliasing does not become a factor in analysis. However, this currently requires offline software processing to reconstruct the original signal (demultiplexing), which can take several hours to compute due to the size of the files. Research speed and experimental significance are limited by restrictions imposed by the acquisition method. A real-time system has been developed to meet the demands of the active flexible electrode array. The system processes and performs analyses on data being recorded from neural tissue via flexible active microelectrode arrays. An experiment is run for several hours and each experiment consists of approximately 30 tests. Each test is approximately 8 minutes long. During an experiment, the data is recorded via a hardware data acquisition system and is read into a software program for processing. The software processes the data in real-time, performs preliminary analyses, and displays the results to provide feedback to the experimentalist. Additionally, at the end of each test, the processed data is saved to disk and offline processing is run on a different computer in parallel with the continuing experiment to provide more detailed analyses of the data. Currently, the system is up and running and in use by researchers in the Litt Lab. The current system does the real-time processing on the CPU (central processing unit) of the data acquisition laptop. The system effectively demultiplexes the data, responds to an event trigger and finds the evoked response of each event. These events are then av-

eraged and visually displayed to researchers who may select target electrodes and channels for observation. Individual electrode evoked responses are collected, processed, and displayed on a color map for researchers to view for the duration of the experiment. The lab has purchased a GPU (graphical processing unit) dedicated desktop to run computationally intensive analyses for a number of concurrent projects. The current system will be used to process the data and display it in real-time and then more computationally intensive analyses will be run on the GPU machine. The remainder of this report will review the related work in the field, present the system model and implementation, review the performance of the system, suggest a plan for the future work on the project, and provide a brief discussion on ethics. 2. RELATED WORK Early work in the field of implantable multielectrode arrays was done by Najafi and Wise [3]. These electrodes had leads that extended out from the circuitry and parts of the leads were exposed to serve as recording points. This work was done with classic rigid silicon electronics and the electronics were housed in a casing near the top of the probe. Mulitplexing, amplification, and buffering of the signal were all performed on chip with hardware. This system recorded from ten channels and the recordings were targeted at single unit recordings. These multielectrode arrays were limited in the number of recording sites, spatial resolution, and the fact that rigid silicon was used which cannot easily conform to biological tissue. In an effort to solve some of these issues, flexible active electronics were developed and this technology was used to make flexible grids of electrodes for recording from biological tissue. These flexible grids could conform to the shape of biological tissue and have very high spatial resolution. In an early generation of flexible active electrode arrays used for biological recordings, Viventi et al. [4] used offline processing to process the data recorded from biological tissue. These electrode arrays were used to obtain cardiac recordings. Cardiac recordings were explored before neural recordings because the heart has significantly larger electrical signals and it is much easier to record signals from the heart than from the brain. The cardiac recordings were used to demonstrate the capabilities of flexible active electronics. The data was acquired with a LabVIEW (National Instruments, Austin, TX) data acquisition system. This system recorded and saved the multiplexed data. Then, this data was converted into a MATLAB readable format and loaded into MATLAB for demultiplexing and processing. This process of converting the data, loading it into MATLAB, and demultiplexing ran at approximately 10 times slower than real time. 12 hours of recordings took approximately 5 days to convert and demultiplex. Kim and Viventi et al [2] used a similar data acquisition system to record brain signals from ultrathin conformal biointegrated electronics. Again, the data needed to be converted and loaded into MATLAB for processing. In this study, recordings were taken from feline visual cortex. Visual stimuli were presented and evoked response averages were calculated for each stimulus type and each electrode. The signals were recorded at 32,556 Hz per channel and down-sampled to 2,713 Hz. The signals were bandpass filtered between 1 Hz and 50 Hz. All of this processing was performed offline. Imfeld et al [1] built a platform to record from 4096 electrodes at 8kHz. This system uses an active pixel sensor (APS) to process the large amounts of data coming off of the array. This processing system implements techniques that were originally developed for image/video processing. Basically, the system treats the electrophysiological data as a sequence of images. Using this technology, high resolution data was processed in real-time. Wagenaar et al [5] developed MeaBench, which is a software toolset for multi-electrode data acquisition and on-line analysis. This software was designed to run on Linux and is not well supported on other platforms. Real-time spike detection, spike validation, burst detection, real-time displays of voltage traces and raster plots as well many other functions are provided functions of the software. Wilson and Williams [6] used GPUs to perform real-time feature extraction for brain-computer interfaces. Using GPUs, which are optimized for parallel processing, they were able to implement computationally intensive algorithms much faster than if they had been just using a conventional CPU. They implemented spatial filtering, auto-regressive power estimation, and classification using the GPUs capabilities. 3. SYSTEM MODEL The experiments for which this system was designed are mainly stimulus-response experiments. In these experiments, an animal has an electrode array placed on a region of brain that is believed to be responsive to a certain type of stimulus. For example, the experiments that will be referenced throughout this paper were performed while recording from feline visual cortex and the stimuli were visual stimuli. The data is recorded by the data acquisition system in segments. The length of these segments can be set by the experimentalist. The data stream from the electrodes is passed through several processing steps before the analyses are presented to the user in generated figures. The first processing step involves demultiplexing the signal and bandpass filtering the signal from 1 to 50 Hz. This produces data for each individual electrode with frequency content in the relevant experimental range. This also removes the DC (direct current) bias which is seen in all of the electrodes and also removes slow drifting of the signal which is also present as a result of the hardware. Subsequently, the trigger signals indicating times of stimulus presentation are processed, and a buffer is constructed containing some number of segments of recorded data. Triggered averaging is computed using provided data. Every time a trigger occurs, (e.g. a stimulus trigger indicating that a stimulus has been presented) the signal is extracted from the buffer from some amount of time before the trigger and some amount of time after the trigger. Extracted segments from each electrode are averaged in real-time, providing characteristic responses of each electrode to a given stimulus. Raw voltage traces provide extremely limited information about the effect of a given stimulus. By averaging the response over a number of trials, error and noise are systematically reduced allowing researchers to observe if meaningful responses were achieved for each electrode. A display is presented and users may select specific electrodes to view both the raw voltage trace and the running average. An example clip of raw voltage input can be seen in Fig. 1. This voltage trace is noisy and not well suited for direct

Figure 1: Half-second Voltage Trace for One Electrode Figure 3: Color Map of Evoked Response Magnitude for Electrode Array Flash at (3,1) 2D Sparse Noise Plot Flash at (8,2) Flash at (6,5) Figure 2: Running Evoked Response Average for One Electrode Figure 4: Visual Explanation of 2D Sparse Noise analysis. As researchers view the incoming data stream, the recorded response on the electrodes looks much like the figure, being updated every half-second in the current experimental protocol. Segments of data like this are stored in the dynamic buffer and combined and averaged to form the evoked response average upon system detection of stimuli triggers. An example reading from the display of the running evoked response average can be seen in Fig. 2. A characteristic response to stimuli can be seen on the figure around sample number 180. For this particular instance, the effective sampling rate of the data was 277 Hz and the signal was averaged for one half second before the stimulus presentation and one full second after the stimulus presentation. Therefore, the stimulus presentation occurs around sample number 138 and the response is seen shortly afterwards. The display allows researchers to observe these responses as they develop and converge to an evoked response in real-time. In real-time, a feature of the evoked responses (a measure of the magnitude of response) is calculated and displayed in a color map. To perform this step, a feature is extracted from each electrode s running average. The magnitude of this feature is displayed by a range of colored pixels on a plot representing the electrode array. While the running average displays significant data for each electrode, it is limited to the scope of one electrode over a duration of time. By aggregating information from all electrodes, researchers can observe the array as a whole to determine if trends occur in the neural response to a given stimulus. Frequently, color maps (Fig. 3) can also provide useful information about the performance of the electrode array. Due to issues that occur during the fabrication of the electrode arrays, arrays which possess single electrodes or entire electrode columns which are nonoperational are commonly seen. These dead electrodes will easily be distinguishable from working electrodes as they will appear as solid black pixels on the color map. This provides researchers with advanced knowledge of nonoperational electronics before offline processing is performed. Furthermore, the color map provides researchers with a view of the overall characteristic response to a given stimulus in real-time for all the electrodes of the array simultaneously. Processed data is also used to generate a figure of the response to the 2D sparse noise experimental paradigm. In this experiment, spots of light are presented randomly at one of 64 potential locations on an 8 by 8 grid. The data is analyzed and evoked responses are computed using the presentations of light at each location as triggers. This analysis can be seen in Fig. 4. In this plot, there is an 8 by 8 grid of plots that represents the areas on the screen where stimuli are presented. At each location, the response of the electrode array is shown for stimuli presented at that location of the grid. In other words, color maps are generated for each position on the 8 by 8 grid. This figure will inform researchers about certain areas of the grid that are responsive to stimuli in a certain part of the visual field. This is an important analysis that shows that the electrode arrays are working properly and are placed on the proper regions of brain tissue. As these experiments are currently performed to test the functionality of the flexible active electronics, providing researchers with a global performance metric in real-time is a powerful tool for successful experimentation. A plot of the response to the 2D sparse noise is generated after some set amount of data reads. This is done because updating the plot after every data read is too computationally intensive and relevant information from this analysis can only be extracted after many data reads.

Save Raw Results Offline Processing (Using Desktop) Frequency Spectrum Power Spectrum Plot and Save Results User Input Legend: Processed Data - LabView C Code Matlab (CPU) Matlab (GPU) - Demultiplexing and Averaging Raw EEG and Event Triggers Data Filtering Online Processing (Using Laptop) Running Average Color Map 2D Sparse Noise Visual Display Save Processed Data Figure 5: System Model After these figures are generated, they are displayed for the user on the visual display. As the system reads in more data, each of the figures is updated to show the response of the neural tissue to the experimental protocol. While this data is being read into the system, it is also concurrently being streamed across the network to a desktop computer dedicated for more intensive analyses and use of GPU computing. There, more complex analyses such as FFTs (Fast Fourier Transforms) and spectral energy content calculations are performed. The remote system also saves a copy of the filtered, demultiplexed data on the desktop machine for further offline analysis. The completed system diagram can be seen in Fig. 5. Each step in the data flow is marked with the platform on which the task was performed. All steps before the branch and located underneath the segment delineated Online Processing were completed in real-time with an actively updating visual display. 4. SYSTEM IMPLEMENTATION The system was implemented using the LabVIEW developer platform, with calls to both C and MATLAB at various stages in the data processing. The functions written in C are called by linking to DLLs (dynamically linked libraries) and the MATLAB functions are called via the MAT- LAB Script node in LabVIEW. LabVIEW was selected for its capability of interfacing between the current and future generation of flexible electronics and its ability to incorporate programs written in various programming languages. A simulated environment was created for the testing of different components of the system. This simulation reads in previously saved data and formats it in the same way that the data is read in by the actual acquisition system. The block diagram for one instance of this simulated environment is shown in Fig. 6. Calls to MATLAB can be seen in this system contained in the script note labeled MAT- LAB script. This block diagram provides insight into the framework of the LabVIEW environment used at the core of the project. Other functions such as variable declarations and visual display interfacing can also be seen in the figure. In the actual data acquisition system (not shown), signals were received from the flexible active electrode arrays using a National Instruments PXI-1033 Chassis along with custom built hardware. After being read in from the electrodes, voltage recordings are read into the LabVIEW software for processing. First stage processing begins when the calls to C code are made to demultiplex and average the data. The data is oversampled and then averaged in order to reduce the noise of the system. The oversampling rate is 20 for the current version of the system, however this can easily be varied. An oversampling rate of 20 means that every point in the final reconstructed signal was constructed by averaging 20 samples. This makes the system robust to computer measurement errors and to high frequency noise fluctuations in the system. Once the signal is averaged, then it is demultiplexed. The current generation of electrodes has 18 rows, so this means that in the averaged signal, every 18th sample for a given channel was generated by the same electrode, and thus the original signals can be reconstructed. For the current generation of flexible electrodes, 20 channels of brain data are fed to LabVIEW, which performs preliminary processing to provide the time domain averaged recordings for each of the 360 electrodes. Each of these channels contains the data for 18 individual microelectrodes due to multiplexing. Five channels of additional experimental parameters, such as triggering times, duration and location are also sent to LabVIEW for later use. All of the numbers that relate to the size of the electrode array are variable. While data is currently recorded using half-second segments, the system has been developed to allow for adjustable sizes of electrode data streams. For the current generation of the data acquisition system, which reads in data at a raw sampling rate of 100 khz per channel and has an effective sampling rate of 277 Hz after demultiplexing and averaging, half-second segments were determined to be reasonable. Filtering is performed as the data is streamed into the LabVIEW software. This is performed by calls to external MATLAB functions which employ a 6th order Butterworth bandpass filter. By processing the data in half-second segments, a great deal of processing can be achieved in realtime. The MATLAB files were modified to be compatible with the data stream. The output of the demultiplexing and filtering steps is passed to C functions which compute evoked response averages for each electrode. The system performs the task by building a buffer that stores the previous 4 half-second segments of data. The size of this buffer is also variable. Upon receiving a trigger, the function takes the buffer and

Figure 6: LabVIEW Block Diagram for Simulated System extracts the relevant piece of the voltage trace. In this case, the relevant piece of the voltage trace was determined to be a half second before the stimulus presentation and a full second after the stimulus presentation. This formed epoch, or period of time surrounding the stimulus that is marked by distinctive features, is then averaged with each previously occurring epoch for that electrode. This data is then passed back to LabVIEW for updating the display and subsequent processing. The evoked response average is then passed to C code which computes a feature for observation in the color map. For each half-second update, if new triggers caused an update of the evoked response average, the features will also be updated. While the current implementation of the system uses simple maximum minimum analysis, integration of new feature analysis is a simple task. A number of features such as RMS (root mean square) and power in frequency bands can be placed into the color map for observation. This feature information is displayed using color map functionality provided by LabVIEW. The color map will refresh upon each update of the selected evoked response average feature. The running average for each electrode is also used in the real-time generation of a 2D sparse noise response plot. Following experimental protocol, flashes of light for unique locations on the 8 by 8 grid provide unique trigger signals. These triggers, along with the evoked response average for each electrode, are sent to a MATLAB function which builds images representing the 2D sparse noise plot. A feature (currently RMS) is calculated for each electrode at each location on the experimental protocol and heat maps are generated for each cell. These are displayed visually using MATLAB s imagesc function and arranged in a way to represent the electrode array. This figure is updated every 50 data reads due to information demands, but can easily be changed as the data stream is modified. A network stream is established between the data acquisition laptop and a desktop computer. As the data is being acquired, it is streamed over the network and saved on the desktop computer for later processing. This desktop!"#"$%&'()*)+,-."/01"/2$345$6$7899 5-:.,(*2$!"#" %&'()*)+,-$."/01"/2 ;3< =;2>,/&2$;?4$@A8B!""#!$%&'!'(')*+,-.!)/01(2/3()!4*5!'*637+,2!0*)-!18!"##9:;&</*,,-7!=1,)017'!6(7>37-?+,2@!015!'-7-<>1, '+2,*7'!A"B!=(.*!=10-'!"ACC!$:;!D01<-''10!=71<9 Figure 7: Hardware Specs and Information computer is equipped with a powerful GPU. The GPU is a GeForce GTX 580. This GPU has 512 Cuda cores and a processor clock speed of 1544 MHz. Analyses are run to determine the frequency spectrum of the recorded signals. Depending on the experiment, the experimentalists can choose from a large selection of possible analyses to run on the desktop machine in parallel with the continuing experiment. The system was developed to interface with specific hardware components and a description of these components can be seen in Fig. 7. 5. RESULTS The system has been used in actual experiments recording from feline visual cortex. In real-time, the system was able to provide feedback to the experimentalists. Fig. 8 gives an analysis of the accomplishments of the system. One of the main improvements of the system over the previous method of analyzing the experimental data is the system s ability to process data as it is acquired. Through use of LabVIEW s ability to interface with C and with MATLAB, the system makes many calls to analysis functions so that this primary analysis is complete by the time the experimental recordings are complete. In one experiment recording from feline visual cortex, the experimentalists were able to see that no region of the grid seemed to be showing an evoked response to a flash of light stimulus. This allowed the experimentalists to change the experimental setup. They abandoned the light stimulus that was currently being used and changed to using the 2D sparse noise paradigm, which proved to be more effective. The feed-

!"#$ %&'()'*&+ 9)"+-6:/8"2" Running time: approx. 10x slower than real-time 8&*.4;<4&=-6: >-42&'-6: %'&,-).#/01#2&* 3&"45!-*&/01#2&* %&'()'*"67& 8&2"-4# %&'()'*"67& 8&2"-4# ~26ms/data read (Run offline after ~80ms/data read (Run offline after Files converted from.tdms to.mat format. Multiple experiments combined (large files) All data demultiplexed at once All data filtered at once?,)$&+/3&#<)6#&/@,&'":-6: ~2sec for entire file Can only be run on entire data file A)4)'/B"<# ~2sec for entire file Can only be run on entire data file C8/0<"'#&/D)-#&/%4)2 ~5sec for entire file Can only be run on entire data file E%F/@6"41#-# N/A GPU analysis not previously performed N/A ~26 ms/data read (Run online concurrently with ~80 ms/data read (Run online concurrently with ~47 ms/data read (Run online concurrently with Online real-time processing does not require data loading Demultiplexed data saved to disk throughout experiment Data filtered as it is acquired. This may cause the filtered data to look slightly different than the filtered data in the previous system. Note: this running time can vary with the number of stimulus triggers per data read. ~100 ms/update Color maps updated in real-time ~500 ms/update Plot updated every 50 data reads in realtime Real-time The GPU is used to perform frequency domain analyses Figure 8: Accomplishments of System back system allowed them to make this change and increase the effectiveness of the experiment and the usability of the results. Along with being used in actual experiments, the system has been tested with simulated data. This is data that was recorded in an experiment and then saved to disk. The simulation software that was developed was used to test the system before it was used in actual experimentation. Fig. 10 shows the results of the 2D sparse noise analysis from a simulated test. This is analysis that was performed offline originally. The simulated system shows a proof of concept that this plot can be generated in real-time to inform the experimentalists of the response of the grid to the stimuli. From this plot, visual fields can be identified and active regions of the grid can be targeted for future experiments. For example, it can be seen from Fig. 10 that the top left corner and the middle of the array responds to stimuli on the grid in row 5, column 4. Therefore, a future experiment could be designed to present stimuli in that region of the grid and look for a response in that region of the array. This type of experimental manipulation and design was not possible without the real-time system. Fig. 9 shows the results of the frequency domain analysis for an electrode from a simulated test. There are a few things to note about this figure. Firstly, there is the characteristic 1/f frequency power distribution, where f is frequency in Hz. This is expected in nearly all naturally occurring signals. Secondly, note the spike at 60 Hz. This is due to 60 Hz environmental noise. This is the frequency at which AC (alternating current) current is provided as electricity in standard outlets and is therefore omnipresent in biological recordings. Additionally, note the spike at 110 Hz. It is less clear what causes this spike, however, it is likely an artifact of the data acquisition system. This type of frequency analysis can be useful in determining a frequency domain response to a given stimulus. Fig. 2 shows the results of the color map analysis of the evoked responses to a stimulus from a simulated test. It can be seen in this figure that the bottom left portion of the array seems to be responding to the stimulus. Again, this information can be used by experimentalists to inform decisions about future experiments. All of these results were compared with the offline analysis that had previously been run on the data to ensure that the results were accurate. 6. FUTURE WORK Future work on this project will consist of adding more functionality to the software. There are additional analyses that would be useful to have during the course of an experiment. One example of such an analysis is real-time spike detection. This would be very useful to see where on the grid spikes were occurring, and this could help correlate spiking activity with clinically relevant information. Additionally, the full capacity of the GPU was not taken advantage of and future work will be to increase the amount of analyses run using GPU computing. Another aspect of the future work for this project will be to run further testing and to fully integrate the system with the experimental design and protocols of the lab. As more experiments are run using the system, more bugs will be

Figure 10: 2D Sparse Noise Plot for a Simulated Experiment discovered and there will be more ideas as how to improve the system. Future work will also involve making full use of the network stream. Data is streamed over the network to a different computer. This additional computer can be used to run many analyses in parallel to the ongoing experiment. This is currently being taken advantage of a bit, but the use of this additional computer could be significantly improved. One more element of the future work of this project is improving the efficiency, readability, and usability of the code and the software system. 7. Figure 9: Power Spectral Density for an Electrode ETHICAL CONSIDERATIONS There are relatively few ethical concerns that arise from the current use of the system. However, there are many foreseeable ethical issues that may arise in future applications of the system. Currently, the system is only being used in animal testing, so the main ethical considerations are for the ethical treatment of animals during the experiments. The proper protocols are followed to ensure minimal suffering of the animals. Anesthetics are administered to the animals so that no pain is felt and the experiments can be performed in the most humane way possible. The animals are also used in conjunction with another neuroscience lab to ensure that the maximal amount of science is performed on each animal. One of the motivations for this project was partially based on this ethical consideration. The system was designed in order to maximize the efficiency and usability of results of each experiment. Additionally, the system is extensively tested on simulated data before it is used in real experiments to work out bugs so that experiments run

smoothly and experimental time is not wasted. Once the system is used in humans, there will be more ethical issues to consider. The system allows real-time analyses of brain signals and therefore information about a person s current brain state can be extracted. As more sophisticated analyses are performed, it will become possible to learn more about what a person is thinking. It will also be possible to identify certain signature patterns of brain activity that map to different cognitive characteristics such as personality type. The ethical issues that arise with this have to do with how this information can be used. As of now, these considerations are theoretical, but will likely become relevant in the future. In human use, the issue of a functional system is very important to ensure that a procedure is performed as efficiently as possible. One of the future uses of the system will be in the mapping of epileptic networks in epilepsy patients to determine if surgery is a viable option. The surgery is quite invasive and involves removing the part of the brain that is believed to be generating seizures. The system must be functioning properly in order to ensure that the correct information is learned about the underlying brain tissue and no unnecessary procedures are performed. mapping cardiac electrophysiology. Science Translational Medicine, 2(24), March 2010. [5] Daniel Wagenaar, Thomas B. DeMarse, and Steve M. Potter. Meabench: A toolset for multi-electrode data acquisition and on-line analysis. Proceedings of the 2nd international IEEE EMBS Conference on Neural Engineering, March 2005. [6] J. Adam Wilson and Justin C. Williams. Massively parallel signal processing using the graphics processing unit for real-time brain-computer interface feature extraction. Frontiers in Neuroengineering, July 2009. 8. CONCLUSIONS The real-time system was successfully developed and implemented. Experimentalists now have a feedback system that assists in the decision making process when modifying an experiment. As was seen in the development and testing of this system, there are many benefits to real-time analyses. The ultimate goal of such a system is to achieve real-time mapping of cortical activity in the brain in order to identify functional and clinically affected neural networks and to use this information to intervene and restore the healthy activity of a patient s brain. 9. REFERENCES [1] Kilian Imfeld, Simon Neukom, Alessandro Maccione, Yannick Bornat, Sergio Martinoia, Pierre-Andre Farine, Milena Koudelka-Hep, and Luca Berdondini. Large-scale, high-resolution data acquisition system for extracellular recording of electrophysiological activity. IEEE Transactions on Biomedical Engineering, 55(8), August 2008. [2] Dae-Hyeong Kim, Jonathan Viventi, Jason J. Amsden, Jianliang Xiao, Lief Vigeland, Yun-Soung Kim, Justin A. Blanco, Bruce Panilaitis, Eric S. Frechette, Diego Contreras, David L. Kaplan, Fiorenzo G. Omenetto, Yonggang Huang, Keh-Chih Hwang, Mitchell R. Zakin, Brian Litt, and John A. Rogers. Dissolvable films of silk fibroin for ultrathin conformal bio-inspired electronics. Nature Materials, April 2010. [3] Khalil Najafi and Kensall D. Wise. Implantable multielectrode array with on-chip signal processing. IEEE International Solid-State Circuits Conference, February 1986. [4] Jon Viventi, Dae-Hyeong Kim, Joshua D. Moss, Yun-Soung Kim, Justin A. Blanco, Nicholas Annetta, Andrew Hicks, Jianliang Xiao, Younggang Huang, David J. Callans, John A. Rogers, and Brian Litt. A conformal, bio-interfaced class of silicon electronics for