Graphics Devices and Visual Perception Human Vision and Perception CRT Displays Liquid Crystal Displays Video Controllers Display Controllers Input Devices Human Vision Eye + Retinal Receptors in eye provide perceptual limits to resolution Human Vision infers properties of world from image: - Shape from shading - Depth from occlusion boundaries - Shape from texture gradients - Shape from stereo - Shape from motion parallax Anatomy of the Eye What is visual perception? Spatio-temporal perception of - Intensity - Colour Human Field of View Monocular - 160 (degrees) horizontal by 135 vertical Binocular - 200 horizontal (h) by 135 vertical (v) c.f. Television 8 h by 6 v - Viewing distance ~ 7.5 times picture height c.f. computer 28 h by 21 v - Viewing distance ~ 2.75 times picture height Spatial Resolution (Rods) Rods 120 million per eye - Black and white (luminance) - Sensitive to low levels of light - Density increases towards centre of visual field, but - Density drops to zero in central vision - (Central 1 region of Retina called fovea)
Spatial Resolution (Cones) Retinal Photoreceptors: Cones 6 million per eye - Three types tuned to: - Red actually tuned to Yellow-Green, peak sensitivity at wavelength = 560nm (L for long), - Green Blue-Green 530nm (M), and - Blue Violet 430nm (S) - Insensitive to low levels of light - Density low away from centre of visual field, but - Density rises dramatically in the fovea Photoreceptor Density Receptors per square millimetre 160 000 120 000 Fovea Blind Spot Rods (B/W) 80 000 40 000 Cones (RGB) 0 70 20 0 20 80 Retinal Eccentricity Foveal Resolution Angular resolution (1 degree corresponds to 1cm viewed at a distance of 57cm): Cones: - 0.5 arcmins (L, M types) - 10 arcmins (S) - Best angular resolution is about 60 cycles per degree - Angular resolution peaks at ~10 cycles per degree Display Resolutions Lines per inch (lpi): number of dark lines plus separating light lines per inch Monitors (CRT & LCD) > 72 lines per inch - at 12 inches viewing distance is 0.07 per pixel Printers capable of 300-1200 lines per inch - 300 lpi at 12 inches is 0.016 (1 arc min) per dot Note that we can resolve ~ 0.017 (1 arc min) Temporal Resolution The temporal frequency at which we no longer see an object flickering is known as the: - Critical Flicker Fusion (CFF) rate CFF varies with spatial frequency and with ambient illumination: - High ambient light, wide field, CFF ~ 80 Hz - Low ambient light, CFF ~ 20 to 30 Hz Temporal v. Spatial Frequency Contrast Sensitivity Temporal Frequency (Hz) 80 60 40 20 Flicker not visible in this region 0 0 20 40 60 80 100 Spatial Frequency (cycles per degree)
Perception of Intensity Illumination intensity Stimulus Response - Perception of intensity Quantisation of intensity Measuring Light and Colour Intensity is defined as power per unit area Radiance is power per wavelength interval Luminance is relative brightness of a source - Compared to standard candle Lightness is a uniform perceptual scale Perception of Intensity Stevens Law related perceived sensation (S) to actual intensity (I): S=I! Typical values for!: Brightness! 0.33 Loudness!0.60 Smell!! 0.60 Heaviness! 1.45 Perceived Brightness Intensity Weber s Law Weber suggested that the Just Noticeable Difference (JND) between two intensities was linearly related to the absolute intensity: JND =!I I " 0.01 Contrast Adaptation to Ambient Light Levels!I/I Human Vision 100:1 Printing 10:1 Monitors 80:1 (at best) Humans perceive about 150 different shades of grey Adaptation allows us to see over very wide range of illumination levels About 100-150 levels
Perceived Grey Level Squares A and B are the same shade of grey! The small squares are all the same colour! Shadow Ilusion created by Ed Adelson from MIT Spectrum CIE Diagram Colour Sensitivity Colour Matching Colour Colour Quantisation Colour Models Electromagnetic Spectrum Newton s discovery that light was made up of colours CIE Chromaticity Diagram CIE Space Co-ordinates in chromaticity diagram represent the relative fractions of each of standard primary colours present in a given colour (x=red, y=green, z=blue). x + y + z = 1 z = 1 - (x + y)
Colour Sensitivity Grassman s Colour Matching Laws Scaling the colour and the component primaries by the same factor preserves colour 2C = 2R + 2G + 2B To match a colour formed by adding two colours add the component primaries (C1 + C2) = (R1 + R2) + (G1 + G2) + (B1 + B2) The retina has 3 types of colour receptors Perceiving Colour Hue - distinguishes between colours Saturation - how far a colour is from a grey of the same intensity Lightness - achromatic perception of perceived intensity reflected from an object Brightness - achromatic perception of perceived intensity from a self-luminous object. Hue and Saturation Vivid Pastel Unsaturated Saturated Distinguishing Colours Side by side hundreds of thousands of colours can be distinguished JND is about 2-10 nm Visible spectrum is ~ 350 nm in extent Can distinguish ~ 128 fully saturated hues Less sensitive to hue changes in less saturated light Energy Dominant Colour Wavelength Pure Colour
Colour Models for Graphics RGB YIQ HSV HLS CMY Cyan Blue RGB Colour Model Black Green White Magenta Yellow Red C = R ˆ R + G ˆ G + B ˆ B RGB Colour Model YIQ Colour Model Used in NTSC TV system (USA) Encodes TV signal as three components: - Y the luminance signal (not yellow!) used for black &white TV (given a radio bandwidth of 4 MHz) - I Orange-Cyan hue information (~1.5MHz) - Q Green-Magenta hue (~ 0.6 MHz) RGB to YIQ conversion HSV Colour Model! Y$! 0.299 0.587 0.144$! R$ # I & = # 0.596 '0.275 Text '0.321&# G& # " Q& # % " 0.212 '0.528 0.311& # %" B& % To reverse operation (YIQ to RGB) use inverse matrix V Green Yellow Cyan White Red Blue Magenta H Black S Hue, Saturation, Value Formed by looking down the diagonal of the RGB cube from white to black. Appeals to artists use of Tint, Shade, and Tone
HSV Hexagon HSV and Tints, Shades and Tones V Tints Tones Shades S Hue, Saturation, Value Cross-section through HSV cone See: 128 Hues 130 Tints (saturation levels) 20 Shades (value levels) > 332800 colours L HLS Colour Model HLS Colour Picker Green Yellow Cyan White Red Blue Magenta Hue is angle about central axis Lightness is distance up central axis Saturation is radial distance from central axis Easy to use, but Non-intuitive in that pure colours are obtained when L=0.5 Black H S Example from Mac OS 9 CMY Colour Model CMY Printing processes operate by using three colour dyes Cyan, Magenta, and Yellow Dyes work by reflection, subtracting light from the incident white light, so colours are specified by what they subtract. Red Magenta Black White Blue Cyan! C $! 1$! R$ # M& = # 1& ' # G& # " Y & # % " 1& # % " B& % Yellow Green
CMYK The combination of dyes C+M+Y should produce Black In practice it usually produces a muddy grey Printers often have a fourth ink that is Black to overcome this problem CRT LCD Graphics Displays Video Controllers Display Controllers Input Devices Immersive Displays CRT Cathode Ray Tube (CRT) Refresh CRT Vector CRT Raster CRT Interlacing Colour CRT Colour Phosphors (patterns and Gamut) Refresh CRT Beam of Electrons focused onto phosphorcoated screen Phosphor emits light when hit by electrons but this fades rapidly (10ms - 60ms) Necessary to redraw the screen rapidly (typically 60 frames per second or higher) Beam can be directed at any point on screen by deflection coils. Key issues are: Refresh CRT Persistence of Phosphor (the time taken for light output to fade - varies with phosphor). Resolution (size of spot on face of CRT - depends on brightness, phosphor, and focussing/deflection system) Typical Resolution 1280 by 1024 (dynamic focussing in best systems [flat screen, high resolution])
Short-Persistence Phosphors P22 Comparison of monochrome fast-phosphors with RGB P22. Critical factor for stereo viewing. Vector CRT Early use of CRT (before memory prices fell enough to make frame buffers economic) Beam is steered at random around screen under command of a graphics processor. Flicker becomes apparent when (typically) many thousand lines are to be displayed on screen Raster CRT Display Raster CRT Display Electron beam sweeps a series of horizontal lines across the screen. Image is stored in a frame buffer memory and output line by line to the CRT synchronised with the raster scan Refresh rates for Graphics CRT s typically ~70 frames per second Interlaced v. Non- Interlaced Raster Television systems draw complete frame as two fields, even lines and odd lines. 25 complete frames of 625 lines per second in UK, i.e. 50 fields per second Screen appears to be refreshed at 50 Hz This trick works if adjacent odd and even lines are similar. Graphics Displays Usually non-interlaced Frame rate > 60 Hz Resolutions of typically 1280 by 1024 Implications for Graphics Controller Hardware Time per pixel is < 12.8 ns
Colour CRT Colour CRT 2 CRT contains three separate systems that scan in close synchrony Three sets of phosphors on inside face of screen and usually three electron beams (slightly displaced from each other) that illuminate the phosphors. Beams are kept separated by shadow masks Phosphor Patterns Colour Gamut Monitor Phosphors! x! y R! 0.635! 0.340 G! 0.305! 0.595 B! 0.155! 0.075 W! 0.313! 0.329 Delta In line Liquid Crystal Displays Structure Operation Passive Active Colour Liquid Crystal Displays (LCD) Developed originally in 1960 s LCD s appearing in watches and calculators in the 1970 s High resolution (1024 by 768) colour LCD displays now dominate laptop computer market (1999) and are competing with CRT s for desktop computer displays
LCD Structure LCD Structure Light is polarised when entering the cell Inner surfaces of glass plates are coated with transparent electrodes Inner most surfaces in contact with liquid crystal treated to force molecules to align themselves parallel to surface at a given orientation Two retaining plates are treated so that their molecular orientations are orthogonal LCD Off State Electric Field is OFF allowing Light Transmission LCD operation OFF state (light transmission): - Liquid Crystal molecules relax and align themselves with treated inner surfaces of front and back electrode plates - Inner surfaces of cell are treated so that one side is orthogonal to the other - Light enters the cell polarised vertically, is rotated by liquid crystal through 90 or more, and passes out through final polarising plate LCD On State LCD Operation Electric field is ON blocking light transmission (dark) ON state (no light transmission, i.e. dark): - Liquid Crystal molecules align themselves with electric field, ie perpendicular to front and back electrode plates - Light enters the cell polarised vertically, is not rotated by liquid crystal, and is thus not able to pass out through final polarising plate
Passive LCD Passive LCD Operation To make dark point at position x, y: - Voltage of -V applied to row y - Voltage of +V applied to column x Individually a voltage of V is insufficient to turn on a cell, but at (x, y) the voltage across the cell is 2V which is sufficient. Cannot use this technique to switch on random points simultaneously. Passive LCD Operation Points can be displayed sequentially row by row (by making each row go to -V in turn). Points along a selected row are made dark by applying +V to selected columns simultaneously. Time constant of liquid crystal molecules is > 100 ms, so once made dark they remain so until refreshed Passive LCD problems! Contrast is poor, grey levels are poor, colour is poor. Time constants (necessary to improve contrast) mean that anything that moves becomes invisible: - In particular, a cursor will disappear when moved (exactly the opposite of what is needed for visual feedback in GUIs), effect known as submarining. Active LCD Operation Transistor at each pixel location Scan circuit enabled by switching on FET transistors in each row in turn. Data for that row is provided to all column wires simultaneously. Voltage for each pixel along a row is stored once FET transistor is switched off by capacitor. Active LCD
Active LCD properties Cells switch from one state to another rapidly ~1 ms to ~10 ms so no submarining effect. Pixels are active all the time: - Brighter display - Higher contrast - Shades of grey easier to display - Better colour Colour LCD Colour display divides each pixel into three sub-pixels Each subpixel is individually controllable and contains liquid crystal material dyed to a particular colour. red green blue LCD Display State of Art LCD Display State of Art (2000) Example Silicon Graphics LCD display: - 17.3 inch diagonal - 16:10 aspect ratio (HD TV) - 1600 by 1024 pixel resolution - 110 dots per inch - 24 bit colour (16.7 million colours) - 120 degree field of view at reading distance - Digital interface IBM T221 (2004) Plasma displays 3840 x 2400 22.2 /22.2 400:1 contrast 5K price tag Active matrix Thin-Flim-Transistor TFT LCD
Giant Plasma TV Sony Thin Film Display 0.3 mm - full colour Panasonic 12ft 6in, HDTV 9M pixel Cost 35K Electronic Paper Display (EPD) EPD Devices Electro Phoretic Display Amazon Kindle Motophone F3 Display Resolution Television NTSC!! 640 x 480 x 8 bits 0.25 Mbytes HDTV!! 1920 x 1080 x 8 bits! ~2 Mbytes Workstation Bitmapped 960 x 1152 x 1 bit ~1 Mbyte Colour W/S! 1280 x 1024 x 24 bits! 5 Mbytes Laserprinter 300 dpi!! US Letter!!! 1.05 Mbytes Film 3000 x 2000!!!!! ~27 Mbytes Increasing LCD Resolution by Sub-pixel Antialiasing Conventional Bit-mapped font Conventional Anti-aliasing of bit-mapped font Using LCD Colours to Triple Horizontal resolution Result of Using LCD Colour triples
Increased Readability Display Bandwidths Media Frames/sec Data rate MBytes/sec Film 24 144 TV 30 8 Workstation 75 375 Simple Raster Graphics Architecture Video Controller CPU System Memory System buses Video controller Display Linear Memory Address System Memory X Address Y Address Pixel Register Raster Scan Generator Intensity Horizontal & Vertical Deflection voltages Typical Raster Display Architecture System Memory CPU Frame Buffer Video controller Display Typical Raster Display Architecture Advantages: - Simple and Cheap Disadvantages: - Software scan conversion - System memory is accessed continuously by graphics controller (maybe 50% of memory bandwidth) - Dual porting Frame buffer helps
RGB Colour Frame Buffer Colour Lookup Frame Buffer Frame Buffer Memory 8 bits per colour DAC DAC DAC Monitor Frame Buffer Memory (8 bits) DAC DAC DAC Monitor Colour Lookup Tables Raster Graphics with Display Controller: Architecture CPU Display Processor Display Proc Memory Frame Buffer System Memory Video controller Host bus Display Display Processors (GPUs) Function is to perform graphics operations in Hardware that would otherwise be done in Software by CPU Restructuring of Graphics pipeline to: - Allow CPU to generate geometry (positions) of objects in the world - Display Processor takes this description and generates pixel values for Frame Buffer Display Processors Typical Functions: - Drawing 2D graphical primitives - Clipping - Drawing 3D objects - Shading - Depth - Texture mapping - Video mixing, decompression Display Processor State of Art NVIDIA GeForce 7800 (2005) 400MHz 38.4 GB/s memory tx rate 10.32 billion texels/second 860 million vertices/second Price: 300 NVIDIA GeForce 8800 (2008) 600MHz 57.6 GB/s memory tx rate 33.6 billion texels/second Stupidly huge vertices/second Price: ~ 350
NVIDIA Human Head Demo Pixel shaders to do refraction of semi transparent objects. Translucence Displacement mapping: volumetric texturing/ bump mapping Real-time hair (geometry for each hair and shadow volumes for better light effects.) Input Devices Locators (data tablets) Trackball Mouse Joystick Light Pen Keyboards Scanners Position Trackers