SPECTRA RESEARCH Institute

Transcription

1 SPECTRA RESEARCH Institte Final Report Neroelectric Activity and Analysis in Spport of Direct Brainwave to Compter Interface Development Richard H. Dickhat prepared for the Office of Naval Research nder Grant #N l-0926 Jne 29, Bena Vista Drive SE, Site 300 * Albqerqe, New Mexico (505) * FAX (505) * Toll Free * spectrar@highfiber.com DTIc QUALITY INSPECTED 1

2 SPECTRA RESEARCH Institte Final Report Neroelectric Activity and Analysis in Spport of Direct Brainwave to Compter Interface Development Richard H. Dickhat prepared for the Office of Naval Research nder Grant #N Jne 29, Bena Vista Drive SE, Site 300 * Albqerqe, New Mexico (505) 243-^^jfcf$&i505) * Toll Free * spectrar@highflber.com

3 REPORT DOCUMENTATION PAGE torn Apprawa OMB No Olnnww TOT wfocwition owfwam»wo w«po 'J^WrtTiioi.»r»«^»m UW-W. www Oftk» <X M^^^tmrw»om»9«t- M*iet»o«rTO(«X(070fc0tW).WMhm<jtoo. OC iojoj. AGENCY USE ONLY {Lttv* bttnk) 2. REPORT DATE 3. «PORT TYPE AND OATE5 COVERED 29IUNE1998 FINAL REPORT 15 JULY 1997 TO 31 MARCH 1998; TITLE AND SUBTITLE 5. FUNDING NUMBERS Neraaleotric Activity and Analysis in spport "of Direct Brair&vave to OcnptEr Interface EeveLcptmiL rauthor(s) Richard H. Didtf)at, EhJ). N PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION REPORT NUMBER I Vectra Research Institte 2201 Bena Vista Dr. SE She. 300 Albxjerqie, KM SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) ' QE ke of Naval Research BaLLstai Centre tcwer One 800 North Qitry Street Arlington, VA SPONSORING/MONITORING AGENCY REPORT NUMBER 1. SUPPLEMENTARY NOTES 12*. DISTRIBUTION / AVAILABILITY STATEMENT 12b. DISTRIBUTION CODE «AHROtfED KR RELIC RELEASE 13. ABSTRACT (Maximm 200 worm) Electroencephalograpic (EEG) scalp sensors were sed as probes to acqire neroelectric data correlated in real time with cognitive processing of lingistic thoght stimli. The adoption of a sitable data analysis methodology provides reslts that show a direct relationship with a stable, qantifiable set of neral activity for digits from zero to nine or the words x yes' 0 r 'no'. SUBJECT TERMS ISfercelectric activity, cognitive processing, lingistic stimli direct htainxave cotmracaticn 7. SECURITY CLASSIFICATION OF REPORT TW* 18. SECURITY CLASSIFICATION Of THIS PAGE 19. SECURTY CLASSIFICATION OF ABSTRACT 15. NUMBER OF PAGES PRICE CODE 20. LIMITATION OF ABSTRAC

4 SPECTRA RESEARCH Institte Neroelectric Activity and Analysis in Spport of Direct Brainwave to Compter Interface Development Richard H. Dickhat, Ph.D. Spectra Research Institte Albqerqe, NM Abstract Electroencephalographic (EEG) scalp sensors were sed as probes to acqire neroelectric data correlated in real time with cognitive processing of lingistic thoght stimli. The adoption of a sitable data analysis methodology provides reslts that show a direct relationship with a characteristic, qantifiable set of neral activity for digits from zero to nine, or the words 'yes' or 'no'. An experimental design was centered on timed captre of single epoch neroelectric data with EEG sensors pon the initiation of thoght stimli. Experimental reslts were analyzed for three test sbjects and twelve lingistic stimli at two electrode locations. The waveform from each location and each single stimls event was sbjected to a decomposition procedre based on the premise that macrocolmns, acting as modlar processing nits, contribte their own simltaneos sbwaveforms to a resltant composite waveform detected by the EEG sensor. The decomposition sbwaveforms analytically obtained are the reslt of an exact soltion of a set of eqations applied to the experimental data for each individal epoch. Each decomposition sbwaveform has its own distinct freqency, amplitde, phase, and decay characteristic. The analytical otcome yields parametric coefficients that determine the characteristics of each sbwaveform. The resltant set of parametric coefficients from a grop of thirty single data epoch waveforms were sed to generate a three dimensional parametric srface that is characteristic and specific for an individal digit or word at a given electrode location for each test sbject. Two electrode locations were chosen that are associated with cognitive, lingistic processing. Each digit or word, then, has a set of 3-D neral activity maps corresponding to the two electrode locations.

5 SPECTRA RESEARCH Institte Backgrond How information is processed, represented, and actively sed in the brain is a major qestion. When the eyes observe a written word, neral spike train seqences are set into play in nerves everywhere along the pathway from the eyes to layers of the neocortex. From there on, immense complexities challenge or capability to model the comptational behavior of large grops of nerons densely interconnected with many thosands of synapses per neron, and any emergence of large scale, complex system behavior. The notion that comptation is involved is attractive becase it is a reasonable hypothesis to assme the brain mst somehow represent information for short time prposes as well as store it for ftre se, and electrical representation of information in a compact form achieved by comptation is a familiar concept. The general strctral characteristics of the neocortex can spply the inspiration and a good basis for model constrction. Neocortical nerons reside in vertical colmns perpendiclar to the cortical srface (Montcastle, 1957,1978; Hbel and Wiesel, 1977; Nata, 1979; Kaas et al., 1981; Gazzaniga, 1989), as well as in six horizontal layers. These colmns have been referred to as macrocolmns, and two decades ago the sggestion was made that they might be modlar processing nits (Rakic, 1976). Two 1996 papers spply sefl insights that cold spport the idea of macrocolmns as modlar processing nits. Trab et al. (1996) describe interneron networks that generate a distribtion of Hz oscillations arising from spike doblets dependent on intracelllar crrent injection. Intragrop connections between networks reslt in coherent oscillations. The paper by Gray and McCormack (1996) introdces a sbset of pyramidal cells in layers 2 and 3 of the visal cortex, called chattering cells (CH), whose stimlation from driving crrent injection reslts in doblet brsts. These experimentally observed Hz oscillations, again in a distribtion, reslt in synchronos firing between cells in different macrocolmns. The CH cells also receive oscillatory brsts in the same range from interneron sorces. The doblet brsts are an effective form of presynaptic inpt - rapid firing can lead to postsynaptic temporal smmation and increase dramatically the probability of presynaptic neron transmitter release. GH cell projections provide a basis for generation of long range synchronos cortical activity. The end srface area of the colmns is small compared to the srface area of a probe of neroelectric brain activity sch as an EEG electrode. Sppose that the information otpt of each of these colmns is more or less independent, bt some redndancy is allowed among grops of closely associated colmns that may be compting similar information content. Let the dynamic change in electrical charge per nit time in these grops of colmns be sensed by the EEG probe over a cortical gyms, and let there be minimal inflence from weaker sorces farther away. The reslt shold be an EEG probe signal representing a composite electrical signatre that reflects the smmation of the signatres from each of the colmn grops more or less directly nderneath the probe. In the case of the cognitive stimli experiments being considered here, the signal response will be referred to as an event-related-potential (ERP).

6 SPECTRA RESEARCH Institte For modeling prposes, a key ingredient is the selection of an appropriate analytical techniqe that can operate on the composite ERP signal and decompose the signatre into its original set of separate waveforms. The techniqe that has been chosen and fond sond in previosly pblished research (Dickhat, 1988) also had two other attribtes that were added to the cognitive model considerations noted above, namely, that the experimental system otpt was prodced by an implse fnction response and that only plse waveform phenomena wold be extracted in the analysis. The first of these two attribtes means that the brain, as the system in qestion, wold be considered to be in an ambient ready state prior to the sdden appearance of a stimls or the prodction of a thoght, and wold then have the capability to respond, as many physical systems wold, to a single exciting stimls (e.g., with peak amplitde and exponential decay phenomena, etc.). In the application of this idea to the brain, it is assmed likely that the stimls response is sbserved by real time dynamics of a mechanism akin to reentrant signaling and information processing as proposed by Edelman (1979). The second attribte means that all continosly valed wave phenomena with the same freqency and amplitde throghot the dration of the electrical signal wold be ignored. The particlar waveform decomposition techniqe chosen presmes that the target ERP signatre is the reslt of an excited strctre responding to implse phenomena, and is composed of mltiple simltaneos waveforms each having its own strctral freqency characteristic. Reslts Experimental reslts were analyzed for three test sbjects and twelve lingistic stimli at two electrode locations that lie directly over cortical areas associated with cognitive processing. The two areas were Brodmann's 39/40 over the left hemisphere; and 0 2 for the right hemisphere. Waveforms were recorded for the resltant neral activity associated with the thoght of a digit ranging from one to nine or the thoght of the words 'yes' or 'no', and sbjected to a special decomposition analysis. For secondary comparison prposes, waveforms were also recorded and analyzed for the experimental condition in which the brain responded to the presentation on a compter screen of the same digits and words. For both experimental conditions, the waveform from each location and each single stimls event was sbjected to a decomposition procedre based on a model of an excited strctre responding to implse fnction neral driving crrents that, in trn, were initiated by the sdden onset of the cognitively meaningfl stimls. A set of sbwaveforms was prodced by the decomposition procedre. Each sbwaveform in the decomposition set has its own distinct freqency, amplitde, phase, and decay characteristic. The next step in the process was to se the analytically-derived parameter coefficients from the sbwaveforms to prodce a 3-D topographical map from the data for each individal event (i.e., physiological voltage activity in the defined time epoch) and stimls type, as well as electrode location, and hman sbject. The 3-D map comprises

7 '«J-CMOeo<0'*C\10CO<D-* <OC0C0<NCMCMCS (\1T--<-T- CM O 00 <D * T- o o o 13 Si c es 9) 'S CS 1H V 3 CUD

8 *(MOIO(D^(MO«0(D fooofocmcmcmcmcm'-t- Th CM O 00 <D t T- O O O 13 c ee a -** 0» 3 IN 3 DC «$TON!&M»4

9 *NO00(D*NO00( )^ OC0C00NI<M(NCMCM-<--«-T- CM O CO CD * T- o o o 1 «GO a» 3 WD

10 «NOIOID^NOOOID^NOaiD^ nomn(\ NNMr;T-^r.T-000 O Ö Ö Ö Ö Ö Ö Ö Ö Ö Ö O Ö Ö Ö Ö -a c 4) OS &

11 *CMOC0<D'<t(\IO00(0 nmonnnmn -a ** 9) X es ft y S 9 ja 1} ; ' ' V'.C V''V.- - ; /i / / */ / a* /a " /iß s. ^K\\\\ö.ssl

12 ^ NOffllO^NO nnrtnnnnn oo CO * CM O 00 CD * T- T- O O O o «3 WD

13 COOrOCMCNlCMCMfVlT-T-T- (M O 00 <D * T- O O O H Iff 3 Xil JD a s on ^v^x^n

14 *CaOOO(D'*(MOOO<D"*CMOOOCD5 nofonnnnnrrrrirooo ÖÖOÖOÖÖÖÖÖÖÖÖÖ 18 o f Ü a> s-

15 *CMO00<DTf(MO00«DTf (M O COCOCOCNCMCMCMCMT-'"-'- O O O O O O O O CO «O * pop odd if. i': c ft CO -** f CO es s

16 «*CMO00(D'*CMO00<D, «l-(mo00<0'* (OnnNNNNNi-i-T-i-riqOO öoooöödooöööooöo 73 o s«m

17 <fr<\10«><0'*t-cmo 00 to * (M O 00 (O * T-; p O O ööööödööö do a s o pa U 85 ft CO cs IT) s- 60

18 *CMOOO<D'«a-CMOeO<DT -<NOCOtD"«d- COCOfO(M(M(MCM<MT-T-^-«-r-000 oödöoödodcidö 4) 'S CO 4) 3 an s

19 ^ (MOOIO'tNO nnonnnnn CO (0 * (M CO <D * O O O a o 3 ü hi 61) s

20 COCOOCMCMCMCMCMT-T-T CM O 00 <D * t- o o o o 'S in V WD e\k\\\\<j.\mfc4

21 *J-CNO00<0TfCNlO00t0'*CMO00<0'*f OOONNNISINrir^'rrqOO oöoööööööooöööoö a^^d.: o U en svvn,\\<i.w!m

22 Tf(MOOO(D'*CMOOOCD'*CM COCOCOCMCMCMCSICMT-T-^TooööddöcJdööö O 00 CD "fr i- o o o a o IS 9 = wo ra^wx^m

23 *{SIO00<0, *(NOC0<D-* COMCO(NCMCM(MCM->-T-T- CM o eo <o * t T o o o öoödööödöööööööö 73 o a 'S 3 CO es B. «Wty^VUM

24 * CM CO CO c* 3 3 S -c 50 <U WD es a 88 o

25 *.N08(0*NOI8(0* focococmcmcmcmcm'r-t-t- CM O 00 td -<fr T- O O O %VT!V\\*^^1

26 fnoomxtnom focoocnolcmcmcmi- (O "tf CM O 00 CO * O O O

27 SPECTRA RESEARCH Institte the parametric data of specific neral activity for given responses to a cognitive stimls and relates the freqencies of the signals activated dring cognition, their intensities (inclding phase information), and the rate at which they decay. The analytical data from approximately thirty epochs of the same stimls type were sfficient to establish a stable, niqe map for each cognitive stimls, electrode location, and test sbject. Figres 1 throgh 7 show examples of 3-D srface maps described above. In each of the figres the x-axis is the freqency in Hertz, the y-axis is the decay coefficient, and the z- axis is the amplitde (inclding phase). Figres with the label 'parietal' in their title correspond to the information obtained from scalp electrodes placed over Brodmann's area 39/40, left hemisphere. Similarly, figres with the label 'occipital' in their title correspond to electrodes over Brodmann's area 0 2, the right hemisphere occipital placement. The primary prpose and reslt of the experiments described here, spported by the comptational analyses, together with the 3-D plots, is the demonstration that individal characteristic neral activity is obtained for specific thoght stimli, electrode locations, and different hman test sbjects. All experiments provided characteristic 3-D maps for all the experimental conditions. They are stable maps; the topological srface featres of each map do not change with increased amonts of data from frther acqisition of experimental epochs for each thoght stimls. It shold be remembered that these maps are the reslt of a mlti-step process of abstraction based on a specific comptational model for cognitive processing described earlier. Each map comprises abot 300 data points (on the average) representing the decomposition parameters for the sbwaveforms for each of the approximately 30 epochs acqired of the neroelectric activity associated with a given stimls. Ths each map is an abstract representation of the neral activity involved in cognitive processing of a specific thoght stimls at a specific electrode location. These parametric maps will form the sfficient basis set of information that will enable a direct identification scheme for evental real time brainwave compter control applications. As a matter of secondary interest, there are a nmber of scientific qestions that can be asked regarding neral activity in cognitive processing with the type of experiments described here and for ftre experiments. As an example, one qestion immediately and always asked in discssing the neral activity maps and cognitive processing is whether there is evidence for similar neral activity between people thinking of the same lingistic items. The presmption in the qestion is that even if totally independent people have different specific brain circitry and langage experience, there has to be some fndamental process/mechanism that enables people, in principle, to commnicate, and perhaps it might be similar neral activity for the concepts and words sed. A second qestion has to do with the possibility of similar cognitive activity in a given individal when thinking of a lingistic item verss responding to an image of it. There is some sggestive evidence in the neral activity map data that both these qestions cold be addressed seriosly with frther refinements in experimental protocol, while still

28 SPECTRA RESEARCH Institte addressing the primary isse of employing real time brain neral activity dring cognition to direct compter fnctions. Discssion The reslts represent a minimm pilot stdy in which EEG ERP data were acqired and analyzed for cognitive processing characteristics related to thinking of the nmbers zero to nine and the words 'yes' and 'no'. The model considerations leading to the experiments have been described above. Stable and characteristic neral activity maps were predicted to be fond, and were fond, for thoght stimli retrieved from memory, jst as they have been for responses to lingistic stimli presented on a compter monitor screen (present experiment, and earlier ones yet to be reported). These reslts comprise a first step in the possibilities for developing a direct brain to compter interface based on real time neral activation related to cognitive processing of specific information. The 3- D maps of abstract parametric data describe a range of neral activity associated with specific lingistic items. The idea is that when sch a map is stored in a compter memory, a sbseqent single epoch of neral activity associated with the same lingistic item can be identified as belonging to that same map, given an analysis of the ERP. The identification of the map can then point to the ASCII word correlate for immediate frther se. The individal epoch ERPs for memory retrieval items were jst as robst as those responses to images of the same lingistic items on a compter monitor. Althogh time zero (to) was known precisely for the image stimli experiment, t 0 identification for the memory retrieval experiment posed a problem. Acqired data after the pixel cross lit p as a time marker on the compter monitor failed to reveal any obvios indication of the start of cognitive processing. More work will have to be done on this problem in ftre experiments as no good, obvios, and direct method was fond to solve the problem for t 0 identification. An indirect method served for the present stdy. Calclation of a time average of the data for all the thoght stimli events for any given stimls nmber for a given test sbject and electrode location, followed by comparing that waveform with the time average waveform for the events prodced by responding to an image of the same stimls nmber, demonstrated that the two waveforms matched in some time history featres for the region between 100 and 200 milliseconds. This was tre for all stimli and electrode locations. Since the image response t 0 was known, the thoght stimls t 0 cold be inferred to be approximately the same, given the time history matching featres. On this basis, a niform t 0 starting point was chosen for all thoght stimli events. Intercomparisons between Figres la, lb, and lc show a similar topological featre among three sbjects in the band of neral activity corresponding to vales of the decay coefficient from 10 to 25. (Note that in these and sbseqent figres the y-scales are often different in different plots dependent on data otcomes from different experiments). The plot for each sbject for the thoght stimls '2' is characteristic for that individal, bt there is a similar 3-D srface featre in the region near the coordinates for 30 Hz and a decay coefficient of 25 for all three sbjects. The featre and srronding topological

29 SPECTRA RESEARCH Institte region sggests that there may be something common in the nerological activity regarding cognitive processing of the nmber '2' among separate individals. Again, there are sggestions for common topological featres in Figres 2a and 2b, for two individals in the instance when the thoght retrieval stimls is '9'. Both Figres 1 and 2 are for the parietal electrode location. Figres 3a, 3b, and 3c show the data for the thoght stimls '9' at the occipital electrode location for the three sbjects. Again, there are hints of common featres to look for in more refined experiments. Remember there is no visal stimls of the nmber '9' involved, bt the visal cortex is showing the evidence for cognitive processing of the thoght stimls. In Figres 4a and 4b, there are sggestions of common featres for a given sbject for both the parietal and occipital location when the thoght stimls is '7'. This was a rare occrrence in this preliminary experiment for different thoght stimls nmbers and sbjects, bt has been observed before in image stimls experiments. Figres 5a and 5b, for a given sbject, show the parietal neral activity data of the thoght stimls experiment verss the image stimls experiment when the stimls nmber is '7'. Figres 5c and 5d do the same for the occipital neral activity data. Given any reasonable featre correlations, the qestion is whether cognitive processing of a thoght stimls has something in common with the cognitive processing of an image stimls, or alternatively, whether perhaps cognitive representation in memory of a lingistic item is always simltaneosly involved as a major factor in real time cognitive processing. Figres 6, 7, and 8 show more comparison data for thoght and image stimls experiments. The analytic approach sed here yields freqencies that are not harmonics and describes only the plse-coded behavior in the original signal while ignoring everything else as backgrond noise. Each analytic reslt is an exact soltion of the specified nmber of simltaneos eqations and conjgate pole pairs in the experimental data. As implemented, the exact soltion is reqired to meet some explicit mathematical criteria, and when this is completed, the otpt lists the parameter vales for the irreglar basis set. The irreglar basis set describes each of the simltaneos decomposition waveforms in terms of their distinct freqency, amplitde, phase, and decay characteristics. The grop waveform set as described above provides a powerfl and qantitative analytic approach to macroscopic electrical signaling prodced by the brain in response to cognitive stimli. As noted, the natre of the ERP signatre has been presmed to be entirely different in strctre than has been classically described. In the most general sense, ERP signatres have previosly been described and analyzed as if they were nitary signals in which seqential brain events dring cognitive processing prodced conseqent and reprodcible time-specific components. The present work does not make the assmption that the ERP is a single nitary signal, bt rather that it is a composite signal that can be decomposed into a finite set of individal waveforms. These waveforms and their parametric coefficients, then, wold be the analytic qantities, rather than the classically named N and P components, that an investigator wold track in

30 SPECTRA RESEARCH Institte cognitive experiments that maniplate physiological or psychological variables as part of the experimental design. The reconstrction of the ERP signatre from the irreglar decomposition basis set shows that N and P components (sch as the P3) are artifacts of the smmation process involving the waveforms. In a few instances, there was a virtal repeat of time domain waveform characteristics starting at arond 350 milliseconds. It was not a consistent observation, bt it might become more common with improved experiments. All processed data sed a time window corresponding to the time period between 50 and 562 milliseconds after the designated t 0. In ftre experiments, there will be an endeavor to improve experimental procedres and compter techniqes for data acqisition in thoght retrieval experiments. In the present experiment the sbject's attention was engaged in the time between the recording of the previos item and the end of the random time interval between 2.5 to 3.5 seconds before the pixel cross lit p on the compter monitor. As reported by the test sbjects, the attention reqirement introdces some element of stress and fatige over the experimental session. One possibility to avoid this problem may be to have the sbject think seqentially of three or for nmbers in a row, then report them. The data acqisition scheme shold be designed clever enogh to pick p ambient data 500 msec, or so before the thoght retrieval of the first nmber. The crrent procedre is also open to the criticism that the sbjects may have rehearsed the items to be thoght of prior to the pixel signal appearance, or otherwise engaged nconsciosly other simltaneos cognitive processing mechanisms, given the instrction to be prepared to think of an item from the stimls set when the pixel signal appeared. The new approach shold help redce possible contamination from these sorces and be more like free flowing thoght prodction. Experimental Design and Procedres The principal experiment was designed to captre physiological voltage signals associated with neocortical information processing as memory retrieval of digits and words took place in a seqence determined by the test sbject. A small comptergenerated cross formed from a few pixels was presented in the center of a compter monitor screen and was the signal for the test sbject to think of a digit from zero to nine or the word 'yes' or 'no'. After the thoght was completed, it was reported to the person handling the data acqisition who then entered the reported item into the compter. In this manner the seqence and the nmber of items in each of the twelve categories were archived for each experiment. The sbjects were instrcted to be prepared to think of a nmber or word from the target list of twelve items as soon as the pixel cross was lit. The instrction was intended to limit the time period in which it wold be necessary to search for indications of the initiation of cognitive processing (or time zero) associated with the thoght activation.

31 SPECTRA RESEARCH Institte The secondary experiment was designed to elicit physiological voltage signals associated with neocortical information processing in response to displayed digits and words on a compter monitor from the same set of items sed in the primary experiment.. A special compter-generated presentation was prepared in which randomly ordered and randomly spaced, yet precisely timed, stimli appeared to the experimental sbjects on a compter monitor. The randomness was achieved by se of a standard mathematical card shffling rotine adapted for the prpose of this experiment. The precise timing enabled correlation with the beginning of the target physiological signal. Each of the twelve stimli appeared forty times The stimli also had random continos pertrbations of pls or mins a half second arond the average stimls separation time of 3 seconds. The test sbject was allowed to temporarily halt the session for a break whenever it was reqested. Six electrodes were placed on each sbject at frontal, parieto-temporal, and occipital locations over both hemispheres, corresponding to Brodmann's areas 46 and 39/40, pls occipital placement O, on the left hemisphere, and mirror image placements on the right hemisphere. Each electrode measrement on a given hemisphere was referenced to a mastoid process electrode location on that side of the head. All recordings are of nipolar signals. The experimental protocol was an instance in which the sbjects were informed and trained sbjects, not naive individals. Each sbject was flly informed abot the design of the experiment, its prposes, and the procedres to be followed. A personal compter (PC) was sed in the preparation and presentation of the pixel timing signal and for the image stimli. The sbjects spent p as mch time as needed with a test compter file that familiarized them with the stimli images, and provided an opportnity to receive training for the short dration interval (600 msec.) of each image stimls presentation. A compter-controlled, mltichannel Grass EEG test instrment was sed to take the EEG data, that were then digitized at a 1 KHz clock rate by a Keithley Instrments analog to digital (A/D) converter, sent to the PC, and finally stored on a hard drive. Over 1.1 seconds of data were recorded in this fashion for each of the six data channels at the onset of each thoght, or stimls presentation, and are permanently archived. The data analysis simltaneosly extracted the freqency, phase, and time-domain information content from the ERP signatre and presented the otpt in the form of a basis set of coefficients that describe the decomposition of the signatre into constitent sbwaveforms. Each of these sbwaveforms is a complex exponential fnction with a distinct amplitde coefficient, niqe freqency, phase, and decay coefficient. Before analysis was ndertaken, each waveform acqired in the experiment was examined for artifacts disrptive to analytical processing, sch as occr in mscle movement, stress, eye blinks, and electrical anomalies. Waveforms with artifacts that wold preclde sccessfl processing were removed from the data set.

32 SPECTRA RESEARCH Institte Waveform Decomposition Analysis The basic assmptions regarding the manifestation of the physiological voltage signal are as given earlier. Additionally, however, the parametric method now being described assmes that the signal contains plse-coded behavior de to the brain's response to a semantic stimls presented dring a special experimental procedre. Under these conditions, the brain's response has been shown to reflect niqe information generating events, and the analysis method shown to decompose sccessflly the composite signal and provide a description of the nderlying plse-coded sbwaveforms (Dickhat, 1988). The response of an electrically active neral strctre to an implse fnction as described in the previos paragraph reslted in a finite sm of complex exponentials, for which a techniqe known as the Singlarity Expansion Method (Bam, 1971; Tesche, 1972) can be applied. Similarly, for N niformly spaced time samples in the signal, it is proposed that the response of the strctre can be written as where A= complex amplitde, 7(0 = I>/ a/ '- (i) 7=1 a = complex natral freqency, n = nmber of independent exponentials in the data, and f,= (i-l)at. It shold be noted that the complex natral freqency does not contain the term 2 n and therefore is a general description not limited jst to harmonic behavior. In the instance in which the transfer fnction of the response is represented by where /i=a zero of the transfer fnction, and F( S )= < s ~ r<k*-r >).. (*-r m _ y ). (s - a,)(j - a 2 )...(s- a ) a i a pole or singlarity of the transfer fnction,

33 SPECTRA RESEARCH Institte the Laplace transform of the transfer fnction yields eqation (1). A partial fraction expansion of eqation (2) yields A F(s)=t^- (3) where Aj is defined as the reside for the j" pole a r Performing an inverse Laplace transform of eqation (3) prodces eqation (1). Ths the complex natral freqencies correspond to poles, and the amplitdes to resides. (Lager et al., 1977). Retrning to eqation (1), sppose that a change in variables has been introdced sch that j k = e a ''. Then if eqation (1) is an eqality for all vales oft, the eqations A ] /i i +A 2 ti 2 +-+A fi =/, A x ri + A 2 {i\+-+a H /i 2 =f 2 (4) A ]M r+a 2M r+-+a,j«r=f N -< are necessarily satisfied, and eqation (1) may be based on the reslts of satisfying these eqations as close as is possible (Hildebrand, 1956). The j 's are also to be determined, reqiring at least 2«eqations, and the eqations are nonlinear in the//'s This difficlty can be minimized by adopting a techniqe known as Prony's method (Prony, 1795). The following description of Prony's method follows that given by Hildebrand (1956). Let //,,..., j n be the roots of the algebraic eqation j" -ay- ] -a 2M "- 2 a n _ xl i-a n = 0 (5) so that the left-hand side of eqation (5) is identified with the prodct (M-Mi)(/~j 2 )---(ß- fi n ). Proceed to modify the eqations in (4) by mltiplying the first eqation by a, the second by <* _,,- -,the ri h eqation by a,, and the {n + \)' h eqation by -1, then add the reslts. Using the fact that each j satisfies (5), the reslt of the addition is in the form /»-«i/ -i a /o=0. (6) A set of TV - (n -1) additional eqations with a similar form can be obtained in the same manner by beginning instead, sccessflly, with the second, third, forth,,(n-n)" 1 eqations. Ths, it is seen that (4) and (5) imply the set of N - n linear eqations 10

34 SPECTRA RESEARCH Institte / _,«, +f - 2 a 2 +-~f 0 a =f f H a \ + f«-\< x 2+-~f l a =/ +1 (7) J N-2 a \ + JN-I a 2 + '"JN-i,-\ a n ~ J N-\. The ordinates f k are known, which means this set generally can be solved directly for the na 's if N = 2«. Once the a's are determined, the «// 's are fond as the roots of (5) and may be real or complex. The eqations in (4) then become linear eqations in the na 's, with known coefficients. The A 's can be determined from the first n of these eqations. Note that the non-linearity of the system has been concentrated in the single algebraic eqation given in (5). The otline jst given provides a smmary of the mathematical essentials of the waveform decomposition methodology. The decomposition otpt comes directly from an exact soltion to the specified nmber of eqations describing the inpt data. References Bam, C.E. (1971). On the singlarity expansion method for the soltion of electromagnetic interaction problems. Air Force Weapons Laboratory, EMP Interaction Note 88. Dickhat, R.H. (1988). Electromagnetic Characteristics of Brain Information Processing: Freqency-coded Responses to Lingistic Stimli. Ann Harbor: University Microfilms International. Edelman, G.M. (1979). Grop selection and phasic reentrant signaling: A theory of higher brain fnction. In: the Nerosciences: Forth Stdy Program. F.O. Schmitt and F.G. Worden eds. Cambridge, MA: MIT Press, pp Gazzaniga, M.S. (1989). Organization of the Hman Brain. Science 245: Gray, CM., and D. A. McCormick (1996). Chattering cells: Sperficial pyramidal nerons contribting to the generation of synchronos oscillations in the visal cortex, Science 274: Hildebrand, F.B. (1956). Introdction to Nmerical Analysis. New York: McGraw-Hill. Hbel, D.H., and T.N. Wiesel (1977). Ferrier Lectre: Fnctional architectre of macaqe monkey visal cortex. Proc. R. Soc. Lond. B. Biol. Sei. 198:

35 SPECTRA RESEARCH Institte Kaas, J.H., R.J. Nelson, M. Sr, and MM. Merzenich (1981). Organization of somatosensory cortex in primates. In: The Organization of the Cerebral Cortex, F.O. Schmitt et äl. eds. Cambridge, MA: MIT Press, pp Lager, D.L., H.G. Hdson, and A.J. Poggio (1977). User's Manal for SEMPEX. A Compter Code for Extrapolating Complex Exponentials from a Time Waveform. Air Force Weapons Laboratory. Technical Report No. TR Montcastle, V.B. (1957). Modality and topographic properties of single nerons of cat's somatic sensory cortex. J. Nerophysiol. 20: Montcastle, V.B. (1978). An organizing principle for cerebral fnction: The nit modle and the distribted system. In: The Mindfl Brain, G.M. Edelman and V.B. Montcastle Cambridge, MA: MIT Press, pp Nata, W.J.H. (1979). Lectre on the association cortex in the session on the organization of the neocortex. Neroscience Research Program (NRP) Cortex Colloqim, NRP Center, Cambridge, MA. Prony, R. (1795). Essai experimental et analytiqe sr les lois de la dilatabilite de flides elastiqes et sr celles de la force expansive de la vaper de 1' alkool a differentes temperatres. J. l'ecole Polytech (Paris):l, Rakic, P. (1976). Differences in the time of origin and in evental distribtion of nerons in areas 17 and 18 of visal cortex in rhess monkey. Exp. Brain Res. Sppl. 1: Tesche, F.M. (1972). On the singlarity expansion method as applied to electromagnetic scattering from thin wires. Air Force Weapons Laboratory, EMP Interaction Note 102. Trab, R.D., Whittington, M.A., Stanford, I.M., and J.G.R. Jefferys (1996). A mechanism for generation of long-range synchronos fast oscillations in the neocortex. Natre 383: