In the name of Allah. the compassionate, the merciful

Transcription

1

2 In the name of Allah the compassionate, the merciful

3 Digital Video Systems S. Kasaei Room: CE 307 Department of Computer Engineering Sharif University of Technology Webpage: Lab. Website:

4 Acknowledgment Most of the slides used in this course have been provided by: Prof. Yao Wang (Polytechnic University, Brooklyn) based on the book: Video Processing & Communications written by: Yao Wang, Jom Ostermann, & Ya-Oin Zhang Prentice Hall, 1 st edition, 2001, ISBN: [SUT Code: TK W ].

5 Chapter 1 Introduction & Basics of Video

6 Outline Color perception & specification Video capture & display Analog raster video Analog TV systems Digital video Kasaei 6

7 Digital Video Processing A video signal is a sequence of 2- D images, projected from a dynamic 3- D scene onto the image plane of a video camera. The color value at any point in a video frame records the emitted (or reflected) light at a particular * 3- D point in the observed scene. * * * n-1 n n+1 Kasaei 7

8 Color Perception & Specification Light -> color perception Human perception of color Type of light sources Trichromatic color mixing theory Specification of color Tristimulus representation Luminance/Chrominance representation Color coordinate conversion Kasaei 8

9 Eye Anatomy From Kasaei 9

10 Eye vs. Camera Camera components Lens Shutter Film Cable to transfer images Eye components Lens, cornea Iris, pupil Retina Optic nerve send the info to the brain Kasaei 10

11 Human Perception of Color Retina contains photo receptors: Cones: day vision, can perceive color tone: Red, green, & blue cones. Different cones have different frequency responses. Tri-receptor theory of color vision [Young1802]. Rods: night vision, perceive brightness only. Color sensation is characterized by: Luminance (brightness). Chrominance: Hue (color tone). Saturation (color purity). From: /retinaframe.html Kasaei 11

12 Human Perception of Color Fig. 1: Simplified diagram of a cross section of the human eye. Kasaei 12

13 Human Perception of Color Fig. 2: Cross section of the eye. Kasaei 13

14 Human Perception of Color Light consists of an electromagnetic wave, with wavelengths in the range of nm, in which the human eye is sensitive. The energy of light is measured in flux, with the unit of the watt. Light is defined as the electromagnetic radiation that stimulates our vision response. It is expressed as a spectral energy distribution. Kasaei 14

15 Human Perception of Color Fig. 3: The electromagnetic spectrum. Kasaei 15

16 Human Perception of Color Fig. 4: Visible wavelengths. Kasaei 16

17 Human Perception of Color Green Red Blue Fig. 5: Typical relative luminous efficiency function of human eye. Kasaei 17

18 Human Perception of Color The luminance of an object is independent of the luminance of the surrounding objects. The (apparent) brightness of an object is the perceived luminance & depends on the luminance of the surround. Kasaei 18

19 Human Perception of Color Fig. 6: Simultaneous contrast. Top: Middle squares have equal luminance but do not appear equally; Bottom: Middle squares appear almost equally bright, but their luminance are different. Kasaei 19

20 Human Perception of Color Human visual system is most sensitive to midfrequencies (3~10 cycles/degree) & least sensitive to high frequencies. Contrast sensitivity also depends on orientation of the grating (max for horizontal & vertical grating). Kasaei 20

21 Human Perception of Color Angular sensitivity variations are within 3dB (Max. deviation at 45 degree). Spatial frequency components, separated by about one octave, can be detected independently by observers. Thus, visual system contains a number of independent spatial channels, each tuned to a different spatial frequency & orientation angle. Kasaei 21

22 Color Representation Use of color is not only more pleasing but it also enables us to receive more visual information. While human can perceive only a few dozen gray levels, have the ability to distinguish between thousands of colors. Kasaei 22

23 Color Representation Fig. 7: Visible color spectrum. Kasaei 23

24 Color Representation Fig. 8: Visible wavelengths. Kasaei 24

25 Color Representation The perceptual attributes of colors are brightness, hue, & saturation. Brightness presents the perceived luminance. Hue refers to its redness, greenness,... Saturation is that aspect of perception that varies most strongly as more white light is added. Kasaei 25

26 Color Representation Fig. 9: Hue representation. Kasaei 26

27 Color Representation Fig. 10: Hue representation. Kasaei 27

28 Color Representation Fig. 11: HSV color model representation. Kasaei 28

29 Color Representation Fig. 12: HSV color model representation. Kasaei 29

30 Color Representation Fig. 13: HSV color model representation. Kasaei 30

31 Color Representation Fig. 14: HSV color model. Kasaei 31

32 Color Representation Fig. 15: HIS color model. Kasaei 32

33 Color Representation Fig. 16: HIS color model. Kasaei 33

34 Color Representation Fig. 17: HIS color model. Kasaei 34

35 Color Representation For monochromatic light sources, differences in hues are manifested by the differences is wavelengths. These definitions are somewhat imprecise. Hue, brightness,& saturation all change when either the wavelength, the intensity, the hue, or amount of white light in a color is changed. Kasaei 35

36 Illuminating & Reflecting Light Illuminating sources: emit light (e.g., the sun, light bulb, TV monitors). perceived color depends on the emitted freq. follows additive rule: R+G+B=White. Reflecting sources: reflect an incoming light (e.g. the color dye, matte surface, cloth). perceived color depends on reflected freq (=emitted freqabsorbed freq.). follows subtractive rule: R+G+B=Black. Kasaei 36

37 Frequency Responses of Cones from [Gonzalez02] Ci = C( λ) ai ( λ) dλ, i = r, g, b, y Kasaei 37

38 Frequency Responses of Cones C i : spectral (radial intensity) response, C(λ) : spectral energy distribution (complete incoming light spectrum), a i (λ) : absorption spectra, (spectral sensitivity curves, frequency responses). Kasaei 38

39 Frequency Responses of Cones & the Luminous Efficiency Function Relative sensitivity Blue 20 Luminosity function Red Green Wavelength Ci = C( λ) ai ( λ) dλ, i = r, g, b, y Kasaei 39

40 Color Hue Specification Kasaei 40

41 Trichromatic Color Mixing Trichromatic color mixing theory: Any color can be obtained by mixing three primary colors with a right proportion. C = TkCk, Tk : Tristimulus values k = 1,2,3 Primary colors for illuminating sources: Red, Green, & Blue (RGB). Color monitor works by exciting red, green, & blue phosphors using separate electronic guns. Kasaei 41

42 Trichromatic Color Mixing Primary colors for reflecting sources (also known as secondary colors): Cyan, Magenta, & Yellow (CMY). Color printer works by using cyan, magenta, yellow, & black (CMYK) dyes. (the dye acts like a narrow-band filter) Kasaei 42

43 RGB vs CMY Kasaei 43

44 red Green Blue Kasaei 44

45 Color Representation Models Specify the tristimulus values associated with the three primary colors: RGB. CMY. Specify the luminance & chrominance: HSI (Hue, saturation, & intensity). YIQ (used in NTSC color TV). YCbCr (used in digital color TV). Amplitude specification: 8 bits for each color component (or 24 bits total for each pixel). Total of 16 million colors. A true RGB color display of size 1Kx1K requires a display buffer memory size of 3 MB. Kasaei 45

46 Color Coordinate Conversion Conversion between different primary sets are linear (3x3 matrix). Conversion between primary and XYZ/YIQ/YUV are also linear. Conversion to LSI/Lab are nonlinear. LSI & Lab coordinates: coordinate Euclidean distance is proportional to actual color difference. Conversion formulae between many color coordinates can be found in [Gonzalez92]. Kasaei 46

47 Color Coordinate Conversion Table 1: Color coordinate systems [Commission Internationale de L Eclairage (CIE)]. Kasaei 47

48 Color Coordinate Conversion Table 1: Color coordinate systems (Cntd). Kasaei 48

49 Color Coordinate Conversion Table 1: Color coordinate systems (Cntd). Kasaei 49

50 Color Coordinate Conversion Table 2: Transformation from NTSC Receiver Primary to other coordinate systems. Kasaei 50

51 Color Coordinate Conversion Kasaei 51

52 Color Coordinate Conversion Kasaei 52

53 Color Coordinate Conversion Kasaei 53

54 Color Spaces CIE XYZ. Kasaei 54

55 Color Spaces CIE XYZ chromaticity diagram. Kasaei 55

56 Color Spaces CIE XYZ chromaticity diagram. Kasaei 56

57 Color Spaces CIE XYZ chromaticity diagram. Kasaei 57

58 Color Spaces CIE XYZ chromaticity diagram. Kasaei 58

59 Color Spaces The RGB safe-color cube. Kasaei 59

60 Color Spaces CIE Lab color models. Kasaei 60

61 Color Spaces color copier. Kasaei 61

62 Color Image Processing RGB color model. Kasaei 62

63 Color Image Processing Pseudo color for detection. Kasaei 63

64 Color Image Processing Pseudo color example. Kasaei 64

65 Color Image Processing Color manipulation. Kasaei 65

66 Video Capture & Display Light reflection physics Imaging operator Color capture Color display Component vs. composite video Kasaei 66

67 Video Capture For natural images we need a light source? (λ: wavelength of the source). E(x, y, z, λ): incident light on a point (x, y, z world coordinates of the point) Each point in the scene has a reflectivity function. r(x, y, z, λ): reflectivity function Light reflects from a point and the reflected light is captured by an imaging device. c(x, y, z, λ) =E(x, y, z, λ) r(x, y, z, λ): reflected light. Courtesy of Onur Guleryuz Kasaei 67

68 More on Video Capture Reflected light to camera: Light intensity distribution in 3-D world: ψ ( X, t) = C( X, t, λ) ac ( λ) dλ Camera spectral absorption function: a Projected image from 3-D to 2-D (video): The projection operator is non-linear: X x P ψ ( P( X), t) = ψ ( X, t) Perspective projection. Othographic projection. or ψ ( x, t) = ψ ( P 1 ( x), t) Kasaei 68

69 Perspective Projection Model Y X Y X 3-D point Z X Z x y F C x y x Camera center X x = F, y = Z Y F Z Image plane 2-D image The image of an object is reversed from its 3-D position. The object appears smaller when it is farther away. Kasaei 69

70 How to Capture Color Single chip color charge-coupled devise (CCD). Kasaei 70

71 How to Capture Color Needs three types of sensors. Complicated digital processing is incorporated in advanced cameras. ( f s,1 ) CCDs f s,1 f s,1 Image 2fs,1 Rate enhancer conv. 2f s,1/f s,2 Digital CN output Lens R B G Analog process A/D Pre-process Interpolation Color corrector Nonlinear processing Matrix & encoder 2f s, MHz D/A D/A Analog CN & CS output Viewfinder output Figure 1.2 Schematic block diagram of a professional color video camera. Reprinted from Y. Hashimoto, M. Yamamoto, and T. Asaida, Cameras and display systems, IEEE (July 1995), 83(7): Copyright 1995 IEEE. Kasaei 71

72 Video Display Monitor phosphor. Kasaei 72

73 Video Display A human observer perceives color through the stimuli of 3 different pigmented cones. Typical absorption spectra of cons in the retina, as a function of wavelength. Kasaei 73

74 Video Display A weighted sum of primaries produces a color that cannot be distinguished by an observer from the color of the spectrum. Additive color model Kasaei 74

75 Video Display Primary & secondary colors of light & pigments. Kasaei 75

76 Video Display Cathode ray tube (CRT) vs liquid crystal display (LCD). CRT needs three light sources projecting red, green, & blue components, respectively. The depth of a CRT needs to be about the same as the width of the screen, for the electrons to reach the side of the screen ( flat CRT). LCD charges the optical properties & consequently the brightness (or color) of a liquid crystal by an applied electric field ( plasma). Kasaei 76

77 Analog Video Video raster Progressive vs. interlaced raster Analog TV systems Kasaei 77

78 Raster Scan Real-world scene is a continuous 3-D signal (temporal, horizontal, vertical). Analog video is stored in the raster format: Sampling in time: consecutive sets of frames. To render motion properly >=30 frame/s is needed. Kasaei 78

79 Raster Scan Sampling in vertical direction: a frame is represented by a set of scan lines. Number of lines depends on maximum vertical frequency & viewing distance (525 lines in the NTSC system). Video-raster: 1-D signal consisting of scan lines from successive frames. Kasaei 79

80 Progressive & Interlaced Scans Progressive Frame Horizontal retrace Interlaced Frame Field 1 Field 2 Vertical retrace Interlaced scan is developed to provide a trade-off between temporal & vertical resolution, for a given, fixed data rate (number of line/sec). Kasaei 80

81 Waveform & Spectrum of an Interlaced Raster Horizontal retrace for first field Vertical retrace from first to second field Vertical retrace from second to third field Blanking level Black level T h White level T l t 2 T T t (a) ( f ) f 0 f Kasaei l 2f l 3f l f max 81 (b)

82 Color TV Broadcasting & Receiving RGB ---> YC1C2 Luminance, Chrominance, Audio Multiplexing Modulation YC1C2 ---> RGB De- Multiplexing De- Modulation Kasaei 82

83 Why not using RGB directly? R,G,B components are correlated: Transmitting R,G,& B components separately is redundant. More efficient use of bandwidth is desired. RGB->YC1C2 transformation: Decorrelating: Y, C1 & C2 are uncorrelated. C1 & C2 require lower bandwidths. Y (luminance) component can be received by B/W recivers. Hue is better retained than saturation. Kasaei 83

84 Why not using RGB directly? YIQ in NTSC I: orange-to-cyan. Q: green-to-purple (human eye is less sensitive). Q can be further bandlimited than I. Phase = Arctan(Q/I) = hue. Magnitude = sqrt (I^2+Q^2) = saturation. Kasaei 84

85 Color Image Y image I image (orange-cyan) Q image (green-purple)

86 I & Q on the Color Circle Q: green-purple I: orange-cyan Kasaei 86

87 Conversion between RGB & YIQ RGB -> YIQ Y = R G B I = R G B Q = R G B YIQ -> RGB R =1.0 Y I Q, G = 1.0 Y I Q, B =1.0 Y I Q. Kasaei 87

88 TV Signal Bandwidth f, : # of active lines. s f, /2 : cycles/picture-height. s y y Maximum frequency that can be rendered properly is lower than this theoretical limit. K=07: Kell factor (attenuation factor). f v,max : Maximum vertical frequency. f h,max : Maximum horizontal frequency. T : Line scanned time (seconds). l f max : Maximum frequency in 1-D raster signal. Kasaei 88

89 TV Signal Bandwidth Luminance: Maximum vertical frequency (cycles/picture-height): black & white lines interlacing: f = v, max Kf ' s, y / 2 Maximum horizontal frequency (cycles/picture-width): f h f, max = v,max IAR Corresponding temporal frequency (cycles/second or Hz): For NTSC: f max = f h,max / T ' l = IAR Kf ' s, y /2T ' l f max = 4.2 MHz Kasaei 89

90 TV Signal Bandwidth Chrominance: Can be bandlimited significantly. I: 1.5 MHz, Q: 0.5 MHz. Kasaei 90

91 Bandwidth of Chrominance Signals Theoretically, for the same line rate, the chromiance signal can have as high frequency as the luminance signal. However, with real video signals, the chrominance component typically changes much slower than luminance. Furthermore, the human eye is less sensitive to changes in chrominance than to changes in luminance. Kasaei 91

92 Bandwidth of Chrominance Signals The human eye is more sensitive to the orange-cyan range (I) (the color of face!) than to green-purple range (Q). The above factors lead to: I: bandlimitted to 1.5 MHz. Q: bandlimitted to 0.5 MHz. Kasaei 92

93 Multiplexing of Luminance & Chrominance Chrominance signal can be bandlimited. It usually has a narrower frequency span than the luminance & the human eye is less sensitive to high frequencies in chrominance. The two chrominance components (I and Q) are multiplexed onto the same sub-carrier using QAM. The upper band of I is limited to 0.5 MHz to avoid interference with audio. Kasaei 93

94 Multiplexing of Luminance & Chrominance Position the bandlimited chrominance at the high end spectrum of the luminance (where the luminance is weak) but still sufficiently lower than the audio (at 4.5 MHz=286 f l ). The actual position should be such that the peaks of chrominance spectrum interlace with those of the luminance: f c 455 f / 2 ( = = l 3.58 Hz for NTSC) Kasaei 94

95 Spectrum Illustration (f ) Luminance Chrominance 0 f l 2f l 3f l 225f l 226f l 227f l 228f l 229f l 230f l f f c (Color subcarrier) Kasaei 95

96 Multiplexing of Luminance, Chrominance & Audio (Composite Video Spectrum) 1.25 MHz 6.0 MHz 4.5 MHz 4.2 MHz 3.58 MHz Luminance I I and Q Audio f p f c f a f Picture carrier Color subcarrier Audio subcarrier Kasaei 96

97 Quadrature Amplitude Modulation (QAM) A method to modulate two signals onto the same carrier frequency, but with 90 o phase shift. s 1( t ) cos( 2 πf 1 t ) m (t ) m (t ) cos( 2 πf 1 t ) LPF s 1( t ) s 2 ( t ) sin( 2πf1t ) sin( 2πf1t ) LPF s 2 ( t ) QAM modulator QAM demodulator Kasaei 97

98 Adding Color Bursts for Synchronization For accurate regeneration of the color sub-carrier signal at the receiver, a color burst signal is added during the horizontal retrace period. Figure from From Grob, Basic Color Television Principles and Servicing, McGraw Hill, 1975, Kasaei 98

99 Multiplexing of Luminance & Chrominance Y(t) LPF MHz I(t) LPF MHz Q(t) LPF MHz -π/2 Σ BPF 2-4.2MHz Σ Composite Video Acos(2πf c t) Gate Color Burst Signal Kasaei 99

100 Demultiplexing of Luminance & Chrominance Composite Video Comb Filter MHz Y(t) _ + Σ LPF MHz I(t) Horizontal Sync Signal Gate 2Acos(2πf c t) -π/2 LPF MHz Q(t) Phase Comparator Voltage Controlled Oscillator Kasaei 100

101 Luminance/Chrominance Separation In low-end TV receivers, a low-pass filter with cut-off frequency at 3MHz is typically used to separate the luminance & chrominance signals. The high frequency part of the I component (2 to 3 MHz) is still retained in the luminance signal. The extracted chrominance components can contain significant luminance signal in a scene with very high frequency (luminance energy is not negligible near f c ). These can lead to color bleeding artifacts. For better quality, a comb filter can be used, which will filter out harmonic peaks correspond to chrominance signals. Kasaei 101

102 What will a monochrome TV see? The monochrome TV receiver uses a LPF with cutoff at 4.2 MHz, & thus will get the composite video (baseband luminance plus the I & Q signals modulated to f c =3.58 MHz). Because the modulated chrominance signal is at very high frequency (227.5 cycles per line), the eye smoothes it out mostly, but there can be artifacts. The LPF in Practical TV receivers have wide transition bands, & the response is already quite low at f c. Kasaei 102

103 Color TV Broadcasting & Receiving RGB ---> YC1C2 Luminance, Chrominance, Audio Multiplexing Modulation YC1C2 ---> RGB De- Multiplexing De- Modulation Kasaei 103

104 Transmitter in More Details Audio FM Modulator 4.5 MHz R(t) G(t) B(t) RGB to YIQ Conversion Y(t) I(t) Q(t) LPF MHz LPF MHz LPF MHz Acos(2πf c t) -π/2 Σ Gate BPF MHz Color Burst Signal Σ VSB To Transmit Antenna Vestigial sideband modulation Kasaei 104

105 Receiver in More Details BPF, MHz Composite video BPF, MHz VSB Demodulator From Antenna Gate Comb Filter MHz 2Acos(2πf c t) Phase Comparator Horizontal Sync Signal FM Demodulator + _ Σ Voltage Controlled Oscillator LPF MHz Kasaei 105 -π/2 LPF MHz Y(t) I(t) Q(t) YIQ to RGB Conversion Audio R(t) G(t) B(t) To Speaker To CRT

106 Matlab Simulation of Mux/Demux We will show the multiplexing/demultiplexing of YIQ process for a real sequence ( mobile calendar ): Original Y,I, Q frames. Converted Y,I, Q raster signals & their respective spectrums. QAM of I & Q: choice of f c, waveform & spectrum. Multiplexing of Y & QAM(I+Q): waveform & spectrum. Kasaei 106

107 Matlab Simulation of Mux/Demux What will a B/W TV receiver see: W/o filtering vs. with filtering. What will a color TV receiver see: Original & recovered Y,I, & Q. Original & recovered color image. Spectrum & waveforms. Kasaei 107

108 Spectrum of Y, I, & Q 10 6 Y Spectrum 10 6 I Spectrum 10 6 Q Spectrum x x x 10 5 Spectrum of Y, I, & Q components, computed from first two progressive frames of mobilcal, 352x240/frame. Maximum possible frequency is 352x240x30/2=1.26 MHz. Notice bandwidths of Y, I, & Q components are 0.8,0.2,0.15 MHz, respectively, if we consider 10^3 as the cut-off magnitude. Kasaei 108

109 QAM of I & Q: Waveform 80 I Waveform 80 Q Waveform 80 QAM multiplexed I & Q Gray Level 0 Gray Level 0 Gray Level Time x Time x Time x 10-4 Line rate f l =30*240; Luminance f max =30*240*352/2*0.7=.89 MHz. The color subcarrier f c =225*f l /2=0.81MHz.. M(t)=I(t)*cos(2πf c t)+q(t)*sin (2πf c t) Kasaei 109

110 QAM of I & Q: Spectrum 10 6 I Spectrum 10 6 Q Spectrum 10 6 QAM I+Q Spectrum x x 10 5 Spectrum of I, Q, & QAM multiplexed I+Q, fc=225*fl/2=0.81 MHz x 10 5 Kasaei 110

111 Composite Video: Waveform 250 Y Waveform 250 Composite Waveform Gray Level 100 Gray Level Time x 10-4 Time x 10-4 Waveform of the Y signal Y(t) & the composite signal V(t)=Y(t)+M(t). 1 line. Kasaei 111

112 Composite Video: Spectrum 10 6 Y Spectrum 10 6 Composite Video Spectrum x 10 5 x 10 5 Kasaei 112

113 Blown-up View of Spectrum 10 6 Composite Spectrum (beginning) Luminance peaks 10 6 Composite Spectrum (near f c ) Chrominance peaks Luminance peaks x x 10 5 Notice that the harmonic peaks of Y & M interleaves near fc. Kasaei 113

114 Composite Video Viewed as a Monochrome Image w/o filtering Original Y Composite Signal as Y On the right is what a B/W receiver will see if no filtering is applied to the baseband video signal. Kasaei 114

115 Low-Pass Filter for Recovering Y Frequency response Impulse response (filter coefficients) Magnitude (db) Frequency (Hz) x Phase (degrees) Frequency (Hz) x f_lpf=30*240/2*150=0.54mhz; fir_length=20; LPF=fir1(fir_length, f_lpf/(fs/2)); Kasaei 115

116 Recovered Y with Filtering Original Y Recovered Y On the right is what a B/W receiver will see if a lowpass filter with cutoff frequency at about 0.75 MHz is applied to the baseband video signal. This is also the recovered Y component by a color receiver if the same filter is used to separate Y & QAM signal. Y (t)=conv(v(t),lpf(t)) Kasaei 116

117 Y Waveform Comparison 250 Y Waveform 250 Composite Waveform 250 Y from Composite using LPF Gray Level Gray Level Gray Level Time x 10-4 Time x 10-4 Time x 10-4 Kasaei 117

118 Demux Y & QAM (I,Q) 80 QAM Waveform 80 Demultiplexed QAM Gray Level 0 Gray Level Time x 10-4 Time x 10-4 M (t)=v(t)-y (t) Kasaei 118

119 QMA Modulation & Demodulation Modulated signal: M(t)=I(t)*cos(2πf c t)+q(t)*sin (2πf c t) Demodulated signal: I (t)=2*m(t)*cos(2πf c t), Q (t)=2*m(t)*sin(2πf c t). I (t) contains I(t) at baseband, as well as I(t) at 2f c & Q(t) at 4f c. A LPF is required to extract I(t). s 1( t ) cos( 2 πf 1 t ) m(t) m(t) cos( 2 πf 1 t ) s 1 ( t) LPF s 2 ( t) sin( 2πf1t ) sin( 2πf1t ) LPF s 2 ( t) QAM modulator QAM demodulator Kasaei 119

120 Low-Pass Filter for Extracting QAM (I+Q) Frequency response Impulse response Magnitude (db) Phase (degrees) Frequency (Hz) x Frequency (Hz) x f_lpf=0.2mhz; fir_length=20; LPF=fir1(fir_length,f_LPF/(Fs/2)); Kasaei 120

121 QAM Demodulation: Waveform 80 Original I 80 Demodulated I 80 Demodulation+LPF I Gray Level 0-20 Gray Level 0-20 Gray Level Time x Time x Time x 10-4 I (t)=2*m(t)*cos(2πf c t) I (t)=conv(i (t),lpf(t)) Kasaei 121

122 QAM Demodultion: Spectrum 10 6 I Spectrum 10 6 Extracted I Spectrum w/o LPF 10 6 Extracted I Spectrum after LPF x x x 10 5 Kasaei 122

123 original I original Q Recovered I Recovered Q Kasaei 123

124 Original color frame Recovered color frame Kasaei 124

125 Different Color TV Systems Parameters NTSC PAL SECAM Field Rate (Hz) (60) Line Number/Frame Line Rate (Line/s) 15,750 15,625 15,625 Color Coordinate YIQ YUV YDbDr Luminance Bandwidth (MHz) / Chrominance Bandwidth (MHz) 1.5(I)/0.5(Q) 1.3(U,V) 1.0 (U,V) Color Subcarrier (MHz) (Db),4.41(Dr) Color Modulation QAM QAM FM Audio Subcarrier / Total Bandwidth (MHz) / Kasaei 125

126 Who uses what? From Kasaei 126

127 Digital Video Digital video by sampling/quantizing analog video raster BT.601 video. Other digital video formats & their applications. Kasaei 127

128 Digitizing A Raster Video Sample the raster waveform = Sample along the horizontal direction. Sampling rate must be chosen properly: For the samples to be aligned vertically, the sampling rate should be multiples of the line rate. Horizontal sampling interval = vertical sampling interval. Total sampling rate equal among different systems: f s = l l 858 f (NTSC) = 864 f (PAL/SECAM) = 13.5 MHz Kasaei 128

129 BT.601* Video Format 858 pels 864 pels 720 pels 720 pels 525 lines 480 lines Active Area 625 lines 576 lines Active Area 122 pel 16 pel 132 pel 12 pel 525/60: 60 field/s 625/50: 50 field/s * BT.601 is formerly known as CCIR601 Kasaei 129

130 RGB <--> YCbCr Y_d = R_d G_d B_d + 16, C_b = R_d G_d B_d + 128, C_r = R_d G_d B_d + 128, R_d = Y_d C_b C_r, G_d = Y_d C_b C_r, B_d = Y_d C_b C_r, Y_d =Y_d -16, C_b =C_b-128, C_r =C_r-128 Kasaei 130

131 YCbCr Format RGB: Y: Cb & Cr (scaled versions of B-Y & R-Y): Cr=240 or R=255, G=B=0 (red) Cr=16 or R=0, G=B=255 (cyan) Cb=240 or B=255, G=R=0 (blue) Cb=16 or B=0, G=R=255 (yellow) Kasaei 131

132 Chrominance Subsampling Formats 4:4:4 For every 2x2 Y Pixels 4 Cb & 4 Cr Pixel (No subsampling) 4:2:2 For every 2x2 Y Pixels 2 Cb & 2 Cr Pixel (Subsampling by 2:1 horizontally only) 4:1:1 For every 4x1 Y Pixels 1Cb & 1 CrPixel (Subsampling by 4:1 horizontally only) 4:2:0 For every 2x2 Y Pixels 1Cb&1CrPixel (Subsampling by 2:1 both horizontally and vertically) Y Pixel Cb and Cr Pixel Kasaei 132

133 Some Abbreviations ITU-R: International Telecommunications Union- Radio Sector ITU-T: International Telecommunications Union- Telecommunications Sector ISO: International Standards Organization SMPTE: Society of Motion Picture & TV Engineers ISDN: Integrated Service Digital Network CIE: Commission Internationale de L Eclariage Kasaei 133

134 Some Abbreviations HDTV: High Definition Television MPEG: Motion Picture Export Group SIF: Source Intermediate Format CIF: Common Intermediate Format IAR: Image Aspect Ratio PAR: Pixel Aspect Ratio DVD: Digital Video Disk VCR: Video Cassette Recorder VOD: Video-on-Demand Kasaei 134

135 Some Abbreviations CCD: Charge-Coupled Device CRT: Cathode Ray Tube LCD: Liquid Crystal Display ATM: Asynchronous Transfer Mode LAN: Local Area Network WAN: Wide Area Network IP: Internet Protocol URL: Universal Resource Locator QoS: Quality of Service Kasaei 135

136 Digital Video Formats Video Format Y Size Color Sampling Frame Rate (Hz) Raw Data Rate (Mbps) HDTV Over air. cable, satellite, MPEG2 video, Mbps SMPTE296M 1280x720 4:2:0 24P/30P/60P 265/332/664 SMPTE295M 1920x1080 4:2:0 24P/30P/60I 597/746/746 Video production, MPEG2, Mbps BT x480/576 4:4:4 60I/50I 249 BT x480/576 4:2:2 60I/50I 166 High quality video distribution (DVD, SDTV), MPEG2, 4-10 Mbps BT x480/576 4:2:0 60I/50I 124 Intermediate quality video distribution (VCD, WWW), MPEG1, 1.5 Mbps SIF 352x240/288 4:2:0 30P/25P 30 Video conferencing over ISDN/Internet, H.261/H.263, Kbps CIF 352x288 4:2:0 30P 37 Video telephony over wired/wireless modem, H.263, Kbps QCIF 176x144 4:2:0 30P 9.1 Kasaei 136

137 Digital Video Applications CIF for video conferences (about half resolution of BT.601 4:2:0). QCIF for videophone (half resolution of CIF). ITU-T H.261 for transport over ISDN (px64 kbps, p=1,,30). ITU-T H.263 for videophone over 28.8 kbpp modem line. Typically: kbps = H.261@64 kbps. Kasaei 137

138 Digital Video Applications SIF for video games and CD movies (a quarter the size of active area in BT.601, about the same as CIF). MPEG-1 for movies on VCDs, compresses SIF 30 mbps to 1.1 mbps (quality the same as VHS VCR). MPEG-2 for high quality broadcasting or on DVDs (HDTV 20 mbps). Kasaei 138

139 Video Terminology Component video Three color components stored/transmitted separately Use either RGB or YIQ (YUV) coordinate New digital video format (YCrCb) Betacam (professional tape recorder) use this format Composite video Convert RGB to YIQ (YUV) Multiplexing YIQ into a single signal Used in most consumer analog video devices S-video Y and C (QAM of I and Q) are stored separately Used in high end consumer video devices High end monitors can take input from all three Kasaei 139

140 Image Fidelity Criteria There are two types of fidelity criteria: subjective & quantitative. Subjective criteria use rating scales such as goodness scales & impairment scales. Quantitative criteria includes: average LSE, MSE, average MS, SNR, PSNR, & frequency weighted MS. Kasaei 140

141 Subjective Criteria Table 3: Image goodness scales. Kasaei 141

142 Subjective Criteria Table 4: Image impairment scales. Sk: score, nk: # observers, n: # grades. Kasaei 142

143 Quantitative Criteria Kasaei 143

144 Quantitative Criteria Kasaei 144

145 Quantitative Criteria Kasaei 145

146 Quantitative Criteria Kasaei 146

147 Quantitative Criteria Kasaei 147

148 Homework Reading assignment: Chap. 1. Problems: Prob Prob Prob Prob Prob Prob Prob Kasaei 148

149 The End