Stimulus presentation using Matlab and Visage

Stimulus presentation using Matlab and Visage

Cambridge Research Systems Visual Stimulus Generator ViSaGe Programmable hardware and software system to present calibrated stimuli using a PC running Windows Accurate frame-based timing e.g. at 130Hz framerate: present stimuli for exactly 7 ms. Enhanced dynamic range 14 bit DAC: 2 14 = 64 times more luminance levels compared to typical 8-bit VGA systems Optimal use of available dynamic range through LUT cycling Gamma correction automatically applied CRS toolbox for Matlab

How does it work? Host PC running Windows controls the external VSG hardware unit (via Matlab or via C/python using win32 API) VSG controls the CRT stimulus display VSG periodically sends a stimulus image to the CRT = refresh cycle One single period of the refresh cycle = frame Rate at which the image is updated = framerate

Refresh cycle Start pixel Horizontal scan line Horizontal retrace Vertical retrace Electron beam High speed exposure snapshot: the electron beam at work in the middle of the screen Horizontal scanning frequency of 96 khz at a resolution of 1280 1024 results in a frame rate of 96,000 / (1024 1.05) 89 Hz time of vertical retrace

Video memory The image sent to the CRT on each refresh is taken from the video memory of the VSG Video memory is subdivided in to video pages

Stimulus representation Each stimulus is represented as a matrix of luminance values Example: the fixation cross Luminance of 1 pixel Video page M = 12 8 12 8 12 8 12 8 12 8 12 8 12 8 1 12 8 12 8 12 8 1 1 1 12 8 812 bit: from 12 black 1 12 to white 12 in 2^8 8 = 256 8 steps 8 8

Drawing stimuli onto the video pages crsclearpage(index, luminance) Clear page and set background luminance crssetdrawpage(index) Choose the video page you want to draw onto crsdrawmatrixpalettised(m) Copy matrix to the video page

global CRS; vsginit; Video page crsclearpage(1,128); crssetdrawpage(1); M = ones(30,30)*128; M(1:30,14:16) = 1; M(14:16,1:30) = 1; Video page crsdrawmatrixpalettised(m);

Predefined drawing functions crsdrawgabor and crsdrawgrating crsdrawline, crsdrawoval, crsdrawstring Other useful functions crspixelstodegrees crspixelstomm crsgetframerate

Help using crs functions crs + [TAB] on the command line for complete overview help command_name

Displaying stimuli Directly using Matlab via crssetdisplaypage: crssetdisplaypage(1); pause(.5) crsclearpage(1,128); Use of this method should be avoided in a multi-tasking OS like windows: many background processes use the CPU This leads to scheduling jitter and dropped frames Inaccurate timing Same problem when using PsychToolbox in Windows

Page cycling Provides accurate timing First, write all your stimuli to the video pages using previously described commands Second, define a page cycle Specify a page number vector denoting sequence of the video page numbers you want to display successively Specify a page frame vector, indicating the number of frames each page has to be presented, indices correspond to indices of the page number vector

Draw stimuli on the video pages 1

Draw stimuli on the video pages 1 2

Draw stimuli on the video pages 1 2 3

Draw stimuli on the video pages 1 2 3 4

Draw stimuli on the video pages 1 2 3 4 5 Press left or right

Define page number vector for 2AFC experiment 1 2 3 4 5 Press left or right Page number vector 1 2 1 3 Present a blank screen Fixation cross Blank again Stimulus 1 1 ISI 4 Stimulus 2 1 Blank 5 Response screen

Define page frame vector 1 2 3 4 5 Press left or right Page 3 will be presented during 2 frames. At 130Hz, presentation time equals 15.4ms Page number vector 1 2 1 3 1 4 1 5 Page frame vector 130 65 130 2 130 2 130 260

Start cycling crspagecyclingsetup(page_number_vector,, page_frame_vector, ) Creates the page cycle, provide page number and frame vectors, along with some additional parameters crssetcommand(crs.cyclepageenable) Executes the cycle At this point, control is transferred to the VSG unit which handles the execution of the cycle stimulus presentation independent of windows accurate timing

Registering subjects responses Accurate RT measurement (1ms precision) using Cedrus RB-530 response box Some useful commands crsresettimer Reset the RT timer crsresponseboxcheck Read from the response box

crsresettimer; [change,state,counter,thestatus] = crsresponseboxcheck; while thestatus==crs.respempty end [change,state,counter,thestatus] = crsresponseboxcheck; ReactionTime = counter/1000000; if state(1)==crs.respdown; end Again, this method leads to inaccurate results in Windows Solution: low-level RTS programming

Real Time Sequencer (RTS) Core of the Visage system Developed to bypass timing issues on multitasking operating systems Scripting engine able to run sequences of operations (i.e., streams ) that are video-frame synchronous Page cycling commands in CRS toolbox for Matlab: highlevel front-end to create, compile and execute an RTS stream that cycles through a series of video pages Reaction time measurement: manual RTS programming is required

How to program the RTS to measure reaction times You can do this easily from within your Matlab experiment code CRS toolbox for Matlab: crsrts-commands First, write all your stimuli to the video pages using previously described commands Second, create and compile and RTS stream (replacing the previous Page Cycling commands) Can be done from within your Matlab experiment code Start the stream

Prepare the draw pages: pg1 = int32(1); pg2 = int32(2); pg3 = int32(3); %---DRAW PAGE 1 : BLANK--- crsclearpage(pg1,127); crssetdrawpage(pg1); %---DRAW PAGE 2 : FIXATION CROSS--- crsclearpage(pg2,127); crssetdrawpage(pg2); M = ones(30,30)*127; M(1:30,14:16) = 256; M(14:16,1:30) = 256; crsdrawmatrixpalettised(m); %---DRAW PAGE 3 : STIMULUS--- crsclearpage(pg3,127); crssetdrawpage(pg3); M = stimulus_matrix(:,:,i); crsdrawmatrixpalettised(m);

RTS_SCRIPT = crsrtscreatestream(0);

RTS_SCRIPT = crsrtscreatestream(0); crsrtsaddstring(rts_script, ' PROGRAM(CyclePages);');

RTS_SCRIPT = crsrtscreatestream(0); crsrtsaddstring(rts_script, ' PROGRAM(CyclePages);'); Predefine variables in the local RTS environment: crsrtsaddstring(rts_script, ' EXTERNAL long pg1 = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long pg2 = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long pg3 = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long prestime1 = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long prestime2 = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long prestime3 = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long resp = 255;'); crsrtsaddstring(rts_script, ' EXTERNAL long ready = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long frcount = 0;'); crsrtsaddstring(rts_script, ' EXTERNAL long RT = 0;');

Present page 1, presentation time = prestime1(in frames): ' SET_VIDEO_WINDOW(pg1,0,0);' ' FOR frcount = 0 TO prestime1 DO' ' BEGIN' ' SUSPEND(SUS_FRAMECOUNT,1);' ' END;'

Present page 1, presentation time = prestime1(in frames): ' SET_VIDEO_WINDOW(pg1,0,0);' ' FOR frcount = 0 TO prestime1 DO' ' BEGIN' ' SUSPEND(SUS_FRAMECOUNT,1);' ' END;' Present page 2, presentation time = prestime2(in frames): ' SET_VIDEO_WINDOW(pg2,0,0);' ' FOR frcount = 0 TO prestime2 DO' ' BEGIN' ' SUSPEND(SUS_FRAMECOUNT,1);' ' END;'

Present page 3 and measure RT: ' SET_VIDEO_WINDOW(pg3,0,0); ' ' FOR frcount = 0 TO prestime3 DO ' ' BEGIN ' ' resp = GET_RESOURCE_VALUE(RT_DIGITAL_INPUTS, 0);' ' ADD(RT, 1); ' ' IF (resp<>255) THEN ' ' ready = 1; ' ' SUSPEND(SUS_FRAMECOUNT,1); ' ' END; ' ' ready = 1;

Present page 3 and measure RT: ' SET_VIDEO_WINDOW(pg3,0,0); ' ' FOR frcount = 0 TO prestime3 DO ' ' BEGIN ' ' resp = GET_RESOURCE_VALUE(RT_DIGITAL_INPUTS, 0);' ' ADD(RT, 1); ' ' IF (resp<>255) THEN ' ' ready = 1; ' ' SUSPEND(SUS_FRAMECOUNT,1); ' ' END; ' ' ready = 1; Compile the stream: crsrtscompilestream(rts_script, 0);

Before executing the stream, we need fill up the predefined variables: crsrtssetstreamvariable('cyclepages', 'prestime1', nr_frames_1, CRS.SS_IMMEDIATE); crsrtssetstreamvariable('cyclepages', 'prestime2', nr_frames_2, CRS.SS_IMMEDIATE); crsrtssetstreamvariable('cyclepages', 'prestime3', nr_frames_3, CRS.SS_IMMEDIATE); crsrtssetstreamvariable('cyclepages', 'pg1', pg1-1, CRS.SS_IMMEDIATE); crsrtssetstreamvariable('cyclepages', 'pg2', pg2-1, CRS.SS_IMMEDIATE); crsrtssetstreamvariable('cyclepages', 'pg3', pg3-1, CRS.SS_IMMEDIATE);

Execute the stream crsrtsstartstream(rts_script, CRS.SS_IMMEDIATE);

Execute the stream crsrtsstartstream(rts_script, CRS.SS_IMMEDIATE); Read values from RTS variables response = int32(0); RT = int32(0); ready = int32(0); while ready == 0 ready = crsrtsgetstreamvariable('cyclepages', 'ready'); end response = crsrtsgetstreamvariable('cyclepages', 'resp'); RT = crsrtsgetstreamvariable('cyclepages', 'RT'); crsrtsdestroystream(rts_script);

Gamma non-linearity Luminance output of most CRT s is not proportional to the voltage of the applied signal but follows a power law L = k (V V 0 ) γ Exact relationship differs between monitors

1 Luminance 0.8 0.6 0.4 0.2 0 Pixel index 1 Luminance 0.8 0.6 0.4 0.2 Pixel index

Power

Gamma correction Ensures accurate reproduction of stimulus contrast or absolute luminance Reproducing images with the same luminance on different monitors and configurations reproduction of experimental data

Gamma correction

Gamma correction using Visage Gamma correction is automatically applied when presenting stimuli using Visage The Gamma nonlinearity is measured using OptiCal in vsgdesktop For color calibration, the ColorCal calibration device is required

Digital-to-analogue conversion Stimuli are generated digitally with very high precision CRT s are analogue devices: luminance controlled by voltage level Typical 8-bit VGA system: 2^8 = 256 luminance levels Effect of limited resolution Again leading to harmonic distortion, especially at low contrasts The smallest luminance step an 8 bit DAC can render is 0.3922% of its nominal range At best, a sine wave grating of.4 -.5% contrast becomes a square wave grating or even a uniform field

Power

White Black 1 2 : 126 127 128 : : 256 Luminance 1 2 3 4 5 6 7 256 DAC value

White Black 1 2 : 126 127 128 : : 256 Luminance 1 2 3 4 5 6 7 256 DAC value

Palette rescaling Ligh t gray Dark gray 126 126. 1 126. 2 : 127 : 127. 8 127. Luminance 9 (Palette) 128 1 2 3 4 5 6 7 256 DAC value

Palette rescaling 25 6 DAC value 1 1 12 8 Luminance 256

6DAC value Combining palette rescaling and gamma correction Actual luminance 50 40 30 20 10 0 Desired luminance 25 1 1 12 8 Luminance 256

Palette rescaling on Visage: LUT cycling L = 0 0. 5 1

Palette rescaling on Visage: LUT cycling L = 0 0. 5 0 0. 5 1 1

Palette rescaling on Visage: LUT cycling L = 0 0. 5 0 0. 5 0. 4 0. 5 1 1 0. 6

Palette rescaling on Visage: LUT cycling L = 0 0. 5 0 0. 5 0. 4 L(:,1) = linspace(0,1,256); L(:,2) = linspace(0,1,256); L(:,3) = linspace(.4,.6,256); 0. 5 1 1 0. 6

Palette rescaling on Visage: LUT cycling L = 0 0. 5 0 0. 5 0. 4 L(:,1) = linspace(0,1,256); L(:,2) = linspace(0,1,256); L(:,3) = linspace(.4,.6,256); 0. 5 1 1 0. 6 Row index indicates page index in the page cycle Element at column i of N denotes the (scaled) voltage level linked to a luminance value i. In case of 8-bit DAC: N = 256

Palette rescaling: LUT cycling crslutbufferwrite Write palette matrix to LUT buffer crslutbuffercyclingsetup Set up the LUT cycle crssetcommand(crs.cyclelutenable + CRS.CYCLEPAGEENABLE); Execute Page cycle and LUT cycle simultaneously