UNIVERSITA DEGLI STUDI DI CATANIA Dipartimento di Ingegneria Elettrica Elettronica e dei Sistemi DIEES Paola Belluomo Tutors: prof. Luigi Fortuna, prof. Maide Bucolo
Brain-Computer Interface (BCI) System Electroencephalography (EEG) Sensory Motor Rhythms Event Related Potential Brain signal acquired Features extraction Features classification New applications - Music composer - Writing robot
Hardware and Software 16 electrodes International 10-20 System - 16 EEG DC-channels - 2 auxiliary DC channels - Interface and power via USB 1.1 cable - Sampling frequency: up to 2000 Hz UDP C++
SMR and BCI2000 application Electroencephalography Sensory-Motor Rhythms(SMR) SMR are electromagnetic oscillations in the frequency range of 8 32 Hz and predominantly originate from the occipital lobe during motor tasks and even during motor imagery. In the field of BCI the deliberate modification of the SMR amplitude during motor imagery can be used to control external applications. Feedback Demo 2 directions: the user imagine to move: -the right arm -the left arm Cursor Task 4 directions: the user imagine to move: -the right arm -the left arm -both the arms -the feet
Application: MUSIC COMPOSER C++ Source code New project creation Running and compiling B A C D E G F -Step 1: the notes are visualized on the screen -Step 2: the targets appear. -Step 3:the cursor appear on the center of the blue screen. -Step 4: the cursor moves according to the signals that derive from the SignalProcessing module (BCI2000). -Step 5: if the ball hit a target, (TargetCode) the corresponding note is placed on the pentagram and the corresponding sound is emitted. Scene modification: -target creation -pentagram -note position FeedbackScene3D::MettiNota( int xpos, int ypos, int inidx ) // xpos posiziona la nota sull'asse x, incrementando ad ogni step di 10 px { mtargets[ inidx ]->setorigin( xpos*10-5, 42.8+yPos*1.2, 50 ); // ypos rappresenta l'indice della nota (da 1 a 7) mpscene->invalidate(); return *this; }
Application: MUSIC COMPOSER
Application: WRITING ROBOT New code INPUT OUTPUT -ResultCode(x,y) -TargetCode= 0 no target hit 1 the ball hit a target SMR -FeedbackDemo.cpp -FeedbackDemoTask.cpp -FeedbackDemoTask UDP CSR BlueCore 4 controller
Robot navigation Three motors A: for the motion of the pen B: right wheel Differential drive model C: left wheel National Instruments LabVIEW Downloading routine for every letter H->scriviH.rxe E->scriviE.rxe L->scriviL.rxe O->scriviO.rxe
Brain-Computer Interface (BCI) System Electroencephalography (EEG) Sensory Motor Rhythms Music composer - More than seven notes sharp (C#, D#, F#, G#, A#) flat (Db, Eb, Gb, Ab, Bb) - New target -> play music Writing robot - Navigation algorithms for all the letters of the alphabet - robot mechanical structure improvement for velocity performance. Brain signal acquired Features extraction Features classification New applications
Brain-Computer Interface (BCI) System Electroencephalography (EEG) Sensory Motor Rhythms Event Related Potential Brain signal acquired Features extraction Non linear analysis for BCI application - Music composer - Writing robot Features classification New applications
Nonlinear analysis for features extraction Training session Thinking to move: 4 stimulus -The right hand -The left hand - Both the hands -The feet Analysis session 1)Filtering Delta: 1-4 Hz Theta: 4-8 Hz Alpha: 8-12 Hz Beta I: 12-16 Hz Beta II: 16-20 Hz Gamma: 20-48 Hz 2)Features extraction 3)Interpretation 1 V n n x 1 ( x ) 2 Performance comparison Integration in BCI system
Brain-Computer Interface (BCI) System Electroencephalography (EEG) Sensory Motor Rhythms Event Related Potential Functional Magnetic Resonance Imaging (fmri) Sensory Motor Rhythms Event Related Potential Non linear analysis for BCI application - Music composer - Writing robot Brain signal acquired Features extraction Features classification New applications
Sistemi complessi adattativi (Prof. L. Fortuna); Attended courses and Tutorials Fondamenti di bioingegneria elettronica (Prof. M. Bucolo); Tecniche di programmazione avanzate (Prof. M. Malgeri); 8-12 Febbraio 2010, Corso di Grid Computing (Consorzio COMETA e Università di Catania); 15-19 Febbraio 2010, Corso di Programmazione Parallela basata su MPI e Grid Computing (Consorzio COMETA e Università di Catania); From 30 August to 3 September 2010, School on Science and technology: knowledge and cultural aggregation (EMUNI University). Conferences X Congresso Nazionale S.I.R.N., Palermo, 22-24 Aprile 2010; XXX Dynamics Days, Bristol, 6-10 Settembre 2010. Scientific Publications P. Belluomo, L. Fortuna, M. Frasca, and A. Sarra Fiore - Implementation of a Wolfram Machine, IEEE Italy Section COMPENG (Complexity in Engineering) Proceedings 2010 February 22-24 2010 Rome, Italy. P. Belluomo, C. Camerano, L. Fortuna, M. Frasca, (2010), From kinetic art to immaterial art through chaotic synchronization, International Journal of Bifurcation and Chaos, 20. P. Belluomo, M. Bucolo, L. Fortuna, M. Frasca, F. Sapuppo(2010), Brain Computer Interface for Control of Dynamics and Robot Systems, XXX Dynamics Days (Europe) Proceedings 2010 6-10 September 2010 Bristol, UK.
THANKS!!!