CS415 Human Computer Interaction

Transcription

1 CS415 Human Computer Interaction Lecture 2 Human Side of HCI Part-1 September 4, 2018 Sam Siewert

2 Things to Remember Assignment #1, Due on Friday CLI Cmd / Rsp Interaction, Command Sequencing, Scripting Chapter 1 in HCI The Human Visual, Auditory, Haptic, Motion Capture (Big 4 for Interaction) Short and Long Term Memory (CLI vs. WIMP) Reasoning, Problem Solving, Skills, Errors Chapter 2 in HCI The Computer Text Input, Pointers, 3D Output Displays touch and stylus VR and Interaction Devices Paper I/O, Memory / Storage, Capacity, Access Processing, Network Lag (client / server delay interactive games), Latency (physical stimulation / response delay, bound by speed of light) Sam Siewert 2

3 Consider What causes humans to miss what should be obvious in visual scenes? [page 21 of text] What is the value of color vision to humans at 1/20 th resolution of grayscale? (6 million cones, 120 million rods) Sam Siewert 3

4 Importance of Color? Often not so important, but sometimes critical [in NAZ] Red on black, you re OK Jack Red on yellow, kill a fellow [1] Seeing Forbidden Colors, Scientific American, pp , Feb [2] How the Color Red Influences Our Behavior, Scientific American MIND, Nov Sam Siewert 4

5 Human Senses and Factors Human Factors in Aviation (Starting with Senses) Classic 5 Senses 1. Visual (Color Tri-stimulus, 2D, 3D), 2. Audio (Tones, Annunciators, Multi-channel Stereo and Surround Sound Speech Recognition), 3. Tactile (Haptics, Ergonomics), 4. Taste (Ignored), 5. Smell (Ignored) Important, but Less Commonly Known: 1. Proprioception Eye to Hand Coordination (VR & AR) 2. Vestibular Balance and Spatial Orientation (VR & AR) 3. Chronoception Time (Interactive and Real-Time Systems) 4. Thermoception (Feeling heat - different from Thermal Imaging) 5. Nociception Pain 6. Interoception Health and Status of Body Organs Sam Siewert 5

6 Human Aspects of HCI James Cutting & Peter Vishton Perceiving Layout Cortex=10 Billion Neurons (High fan-out) > 1 Trillion Synapses Total=100 Billion Neurons 1. Neuron > Transistor 2. Better Programming? ROM? 3. More Richly Interconnected 4. Storage + Processing Red Epic Mega-Pixel Approximately 100+ Mega-Pixel (Rod/Cone Count) 6-7M Cones Neuroscience. 2nd edition. Purves D, Augustine GJ, Fitzpatrick D, et al., editors. Sunderland (MA): Sinauer Associates; CPU Local Bus 5 To 7 billion transistors CPU Memory Controller I/O Bus (x16 5Gbps = 8GB/sec) Camera Link Interface Card Sam Siewert 6

7 Perception Driven by Expectation (Visual Cues + Mental Model) Our Model of the World is as Important as Our View Aviation Visual Perception - Flight into Terrain Research, Misperception and Mishaps Specular Reflection M.C. Escher Master of Use of Misperception In Art Wikipedia - Perception is not only the passive receipt of these signals, but it's also shaped by the recipient's learning, memory, expectation, and attention. [4][5] Gibson, James J. "The perception of the visual world." (1950). [Described optical flow] Sam Siewert 7

8 Optical Illusions the Ponzo illusion the Muller Lyer illusion

9 Visual Perception is Complex 8 or More Visual Cues for Human Depth Perception Relative Importance of Cues (Context Driven) Binocular disparity Relative size Motion perspective Height in visual field Occlusion Accommodation Aerial perspective (Contrast decreases, Rayleigh Scattering) Texture gradient Sam Siewert 9

10 Movement (cont) Fitts' Law describes the time taken to hit a screen target: Mt = a + b log 2 (D/S + 1) where: a and b are empirically determined constants Mt is movement time D is Distance S is Size of target targets as large as possible distances as small as possible

11 Visual Cues Depend Upon Space (Scene) of Interest James Cutting & Peter Vishton Perceiving Layout Sam Siewert 11

12 Visual Computing Graphics Forward Rendering From Scene Description to Image RGB Resolution and Bits per Color Channel Computer Vision Inverse Rendering From Image to Scene Description Light Field, Not Limited to Visible Photometry Human Visual Cues Characterized but Far From Fully Understood Virtual Reality, Augmented Reality (Challenges) Sam Siewert 12

13 Radiometry vs. Photometry Radiometry the study of light from viewpoint of Physics Energy (joules), Flux (photons/cm 2 ), Power (watts/m) Full Electromagnetic Spectrum including Visible from Far Infrared to Gammarays Photometry study of light from perspective of useful, often visible light Sam Siewert 13

14 Photometry What Does Eye See? Color Models RGB Cube HSV - Hue/Saturation/Value Hue Similarity to R, G, Y, B Saturation Intensity of Specific Color Value Low=Black, High=Color Luminance (Candela/Square-Meter) Light Passing Through Area Forming a Solid Angle in A Direction Candela (Photonic Power )= Watts/Steradian More Precise than Brightness Chrominance ( CrCb or UV in YCrCb or YUV) U=Blue Luminance (Y) V=Red - Luminance (Y) Wavelength Spectrum - ROYGBIV RGB Cube HSV Cylinder/Cone saturation value hue Sam Siewert 14

15 PIXEL: srgb Basic Color Model Basic Encoding of Picture Element (Pixel) in Grayscale or Color RGB, 24-bit, 8 bits [0-255] for each color band (x, y, z) sampled Each Pixel is a 3-D Vector in RGB Space, Opponent Colors Based On Additive Color Model Compared to Printer s Subtractive Model CYMK Emission of Light in Color Bands (CRT, LCD, etc.) Colorimetry science of color measurement and response a human response to the visible spectrum based on physiology Magenta Blue White Cyan Black Green Bayer 2x2 Pixel Example Red Yellow Sam Siewert 15

16 Conversion from RGB to HSV Two Simple Color Models saturation value hue Sam Siewert 16

17 Machine Vision Color in Brief Physics of Wavelength, Intensity of Electromagnetic Radiation Luminance (Y), Chrominance U=B-Y (blue-luma), V=R-Y (red-luma) Additive Primary Hues (R, G, B) Secondary Hues (Combined R+G, R+B, G+B) CMY Complete subset of Colors is Gamut Chromaticity is independent of the luminance Additive model Measured in Visible for Classic Photometry (UV and Infrared) Visible Photometer (detectors sensitive to intensity in λ bands) Luminous flux compared to standard source (lumen) visible to eye Number of photons or flux in photons/cm2 Spectrograph (frequency separation from incident wave) Radiometry or Radiant flux total power in watts Subtractive model Physiology of What We Can See Tristimulus L ( nm) S ( nm) M ( nm) of Photoreceptors What We See in Various Lighting Conditions Different and Perhaps Not Identical to the Photometer Perhaps Not Identical to Display in Lighting Conditions Tri-Stimulus (Human) Encoding from Measurements to Digital Samples Per Pixel, by color band, in an array Sam Siewert 17

18 Display Color Transformation Adapting Rec 709 srgb Color Model to Different Lighting Conditions Cinematic, Ambient Fluorescent or Tungsten, Full-spectrum sunlight Based on Primaries, Tri-stimulus, and Lighting, We See a different Gamut Rec 709 Color Space And D65 White Point Plot of Colors Visible to Eye CRT Gamut and CIE Chromaticity Diagram Chromaticity Diagram L ( nm) S ( nm) M ( nm) Sam Siewert 18

19 Color Adaptation to Ambient Lighting E.g. Work with eecolor for RT Color Transform at 1080p Gamut Boost for Lighting Conditions and Display Characteristics Sam Siewert 19

20 What about Grayscales? The Eye Does Not See gray levels lineraly Gray levels Designed for the Eye E.g. Gamma Correction Tri-stimulus of Eye Follows gamma power function Gamma FAQ Color FAQ Panchromatic Sam Siewert 20

21 YUV ITU-R BT.601 Component Video Standard Y is Luma UV is Color information designed so that BW TV (CRT) will still display grayscale image Used by NTSC, PAL, SECAM (Standard Definition TV) RGB to YUV Conversion Y = (0.299 * R) + (0.587 * G) + (0.114 * B) U = -(0.147 * R) - (0.289 * G) + (0.436 * B) V = (0.615 * R) - (0.515 * G) - (0.100 * B) YUV to RGB Conversion R = Y V G = Y (0.394 * U) (0.581 * V) B = Y + (2.032 * U) Sam Siewert 21

22 POINT: R, G, or B band only vs. Linear Combination (Human Balance) R G B Y = R + G=R + B=R Y = R=G + G + B=G Y = R=B + G=B + B Balanced Y = 0.3R G B Sam Siewert 22

23 OpenCV Single Channel - Balanced 3 Shades of Gray, from Color Channels split out of BGR Code for Windows is here Sam Siewert 23

24 OpenCV Single Channel - Unbalanced 3 Hues - B+zero(GR), G+zero(BR), R+zero(BG) Sam Siewert 24

25 POINT: YCrCb RGB An Alternative to RGB is YUV, Where Y is Luminance and CrCb is Chrominance The following 2 sets of formulae are taken from information from Keith Jack's excellent book "Video Demystified" (ISBN ). RGB to YCrCb Conversion (For Computers with RGB [0-255]) Y = (0.257 * R) + (0.504 * G) + (0.098 * B) + 16 Cr = (0.439 * R) - (0.368 * G) - (0.071 * B) Cb = -(0.148 * R) - (0.291 * G) + (0.439 * B) YCrCb to RGB Conversion R = 1.164(Y - 16) (Cr - 128) G = 1.164(Y - 16) (Cr - 128) (Cb - 128) B = 1.164(Y - 16) (Cb - 128) In both these cases, you have to clamp the output values to keep them in the [0-255] range. Sam Siewert 25

26 POINT: YCrCb 4:4:4 24-bit Format For every Y sample in a scan-line, there is also one CrCb sample Each Y (Y7:Y0), Cr (Cr7:Cr0), & Cb (Cb7:Cb0) Sample is 8 bits No compression between RGB and YCrCb 4:4: ,480 76,799 = Y, Cr, and Cb sample = Y sample only Sam Siewert 26

27 POINT: YCrCb 4:2:2 16-bit Format For every 2 Y samples in a scan-line, there is one CrCb sample bit to 32 bit 76,480 76,799 = Y, Cr, and Cb sample = Y sample only Pixel-0 = Y7:Y0 0, Cb7:Cb0 0 ; Pixel-1 = Y7:Y0 1, Cr7:Cr0 0 Pixel-2 = Y7:Y0 2, Cb7:Cb0 1 ; Pixel-3 = Y7:Y0 3, Cr7:Cr0 1 Pixel-4 = Y7:Y0 4, Cb7:Cb0 2 ; Pixel-5 = Y7:Y0 5, Cr7:Cr0 2 Sam Siewert 27

28 FRAME: XY Pixel Maps FRAME Resolution Computer Graphics Resolutions VGA = 640x480 SVGA=800x600 NTSC = Standard Defintion, 720x480 High Definition (Progressive full frame or Interlaced) 720i/p = 1280x i/p = 1920x1080 FRAME Aspect Ratios X to Y Ratio NTSC = 3:2, 720x480 (240) HD 720 = 16:9, 1280x720 (80) HD 1080 = 16:9, 1920x1080 (120) 2K = 17:9, 2048x1080 (120) FRAME Rates 60i, NTSC = odd/even, or p, Cinema = 24 fps (motion blur) 60p, HDTV Progressive Modes Sam Siewert 28

29 Display and Camera Resolutions Red Epic 645 9K: 9334x7000, Red Epic K: 28000x9334 Sam Siewert 29

30 Perspective Studies by Artists and Physics of Light Silk Screen Painted/Traced Edge Illumination Observer Viewpoint Rays Sam Siewert 30

31 Perspective Concept Object Render Plane Intersections viewpoint Sam Siewert 31

32 Parallel Projection Cartography Orthographic Mercator (Cylindrical) Orthographic Sam Siewert 32

33 Vanishing Point Linear Perspective (1 to 3 Vanishing Points) Curvilinear (4 or 5 Vanishing Points) Single Vanishing Point Double Vanishing Point Sam Siewert 33

34 Perspective Projection Projection Method Used in Ray Tracing and Rendering Define Extents of Render Plane l=left, r=right, t=top, b=bottom Viewpoint e is shown on Render Plane Move e out of Plane to Location of Camera/Viewer Fundamentals of Computer Graphics, 3 rd Edition - page 75 Sam Siewert 34

35 Short-term memory (STM) Scratch-pad for temporary recall rapid access ~ 70ms rapid decay ~ 200ms limited capacity - 7± 2 chunks

36 Long-term memory (LTM) Repository for all our knowledge slow access ~ 1/10 second slow decay, if any huge or unlimited capacity Two types episodic serial memory of events semantic structured memory of facts,concepts, skills semantic LTM derived from episodic LTM