Intro. To Multimedia Engineering Slide 4 - Fundamental Concepts of Video

Transcription

1 Intro. To Multimedia Engineering Slide 4 - Fundamental Concepts of Video Kyoungro Yoon yoonk@konkuk.ac.kr 1/31

2 Contents Analog Video Digital Video Types of Video Interfaces 3D TV Mainly From Chapter 5 of Fundamentals of Multimedia by Ze-Nian Li and Mark S. Drew 2/31

3 Brief History of TV 1883, German Engineering Student Paul Nipkow applied for a patent which enables TV using image scan. Jan. 26, 1926, first TV broadcasting test at a department store in London by John Logie Baird 1929, First TV studio opened by John Baird 1930, first regular TV broadcast by BBC, UK. 1940, First color TV with 343 line resolution invented by Peter Goldmark 1941, FCC published NTSC B/WTV standard Getting popular after /31

4 Analog Video Scanning Begins at the upper left corner and moves horizontally across the image, making a horizontal scanning line. The horizontal line moves down slowly as it moves right. When the right side of the image is reached, the scanning point moves back to the left side of the image and scans the next line. When the scanning point reaches the bottom right corner, it moves to the top right corner of the image and scanning of next image starts. Horizontal Blanking Interval / Horizontal Retrace: The time interval that the scanning point moves from the right edge to the left edge of the image. Vertical Blanking Interval / Vertical Retrace: The time interval that the scanning point moves from the bottom right corner to the top left corner. (The temporal gap between frames.) Sync Information: Signal that a camera generate to ensure the display scanning is in synchronism with the sensor s scanning in the camera. 4/31

5 Analog Video Progressive Scanning Traces through a complete picture (frame) row-wise for each time interval. Computer monitors typically use the time interval of 1/72 seconds (72Hz). Interlaced Scanning Odd-numbered lines are traced first and then even-numbered lines are traced. Odd numbered lines constitutes odd fields and even numbered lines constitutes even fields. Two fields make up one frame. 5/31

6 Analog Video Interlaced scanning Fluorescent phosphors in the CRT emits lights for very short time. The image on the display disappears unless it is redisplayed (refreshed) after this short period. If the refreshing interval is too large, flickering effect appears. The frame rate should be at least 50 Hz. To avoid flickering effects in 25 fps / 30 fps systems, interlaced scanning is used. 6/31

7 Analog Video Sync Signal Tells the system to begin a new line. Use offset from zero to indicate the start of a line. Called blacker-than-black zero signal. White has a peak value of V (NTSC) Black is at V, slight above zero. Blank is at 0 V. Sync is at V 7/31

8 Analog Video NTSC Video TV Standard for North America, Japan, Korea 4:3 Aspect Ratio / 525 lines/frame / frames/sec / 33.37ms/frame Interlaced scanning system : lines/field Horizontal sweep frequency: 525x29.97 => lines/sec Each line is swept out in 1/15734 = 63.6 msec. Horizontal retrace for 10.9 msec msec for active signal. Blanking information: 20 lines reserved for control information at the beginning of each field. => Number of active lines per frame is 485. Non-blanking pixels are called active pixels. 8/31

9 Analog Video NTSC video is an analog signal with no fixed horizontal resolution. Therefore one must decide how many times to sample the signal for display: each sample corresponds to one pixel output. A pixel clock is used to divide each horizontal line of video into samples. The higher the frequency of the pixel clock, the more samples per line there are. Different video formats provide different numbers of samples per line, as listed in Table /31

10 Analog Video Samples per line for each format VHS 240 S-VHS Betamax 500 Standard 8 mm 300 Hi-8 mm /31

11 Analog Video NTSC uses YIQ color model. Quadrature Modulation Combine I (in-phase) and Q (quadrature) signals into a single chroma signal C. C=Icos(F SC t)+qsin(f SC t) : Also known as color subcarrier. 2 2 Magnitude Phase I Q tan 1 ( Q / I) Frequency of C: F SC ~~ 3.58 MHz Composite = Y + C = Y + Icos(F SC t)+qsin(f SC t) 4.2 MHz for Y, 1.6 MHz for I and 0.6 MHz for Q 11/31

12 Analog Video NTSC video channel: 6MHz Picture Carrier: at 1.25MHz in the NTSC Video Channel Chroma signal carried by FSC ~ 3.58MHz MHz = 4.83MHz to reduce potential interference between Y and C signals 15/31

13 Analog Video NTSC signal decoding 1. Separating Y and C - Low-pass filter to extract Y at the lower end of the channel. - Use comb filters to extract interleaved Y signal from the higher end of the channel. 2. Extract I 1. Multiply C by 2cos(F SC t) C2cos( F SC t) I 2cos 2 ( F SC t) Q2sin( F SC t)cos( F SC t) I(1 cos(2f SC t)) Q2sin( F SC t)cos( F SC t) I I cos(2f SC t) Qsin(2F SC t) 2. Apply low-pass filter to obtain I and discard the two higher-frequency 2F SC terms. 3. Extract Q 1. Multiply C by 2sin(F SC t) 2. Apply low-pass filter to obtain Q. 13/31

14 Analog Video NTSC color TV at the frame rate 30 x 1000/1001 ~ fps NTSC color subcarrier frequency lowered to f SC = 30 x 1000/1001 x 525 x ~ MHz Where is the number of color samples per scan line. 14/31

15 Analog Video PAL (Phase Alternating Line) Video TV standard for Western Europe, China, India, Australia, etc. 625 scan lines per frame 25 frames per second (40 msec/frame) 4:3 aspect ratio with interlaced fields Composite Color using YUV color Model 8 MHz channel w/ 5.5 MHz to Y, 1.8 MHz each to U, V Color subcarrier f SC ~ 4.43 MHz Chroma signals have alternate signs in successive scan lines +U and U Phase alternating line Signals I consecutive lines are averaged to cancel the chroma signals and separating Y and C obtaining high-quality Y 15/31

16 Analog Video SECAM (Systeme Electronique Couleur Avec Memoire) Video 625 scan lines per frame 25 frames per second 4:3 Aspect Ratio and Interlaced fields Similar to PAL but different color coding scheme Color subcarrier for U: 4.25 MHz Color subcarrier for V: 4.41 MHz Color signals sent in alternate lines. (One of U or V for each scan line) 16/31

17 Analog Video Comparison of Analog Broadcast TV System TV System Frame Rate (fps) Number of scan lines Total Channel width (MHz) Bandwidth allocation (MHz) Y I or U Q or V NTSC PAL SECAM /31

18 Digital Video Advantages of Digital Representation Storing Video on Digital Devices (HDD, Memory) w/o losing quality Ready to be processed (noise removal, editing, etc) Integrating in various MM applications Direct access making nonlinear video editing simple Repeated recording w/o degradation of image quality Ease of encryption and better tolerance to channel noise. Started as a composite video, but evolved to component video using RGB / YUV / YCbCr 18/31

19 Digital Video Chroma Subsampling Human see color with much less spatial resolution than black and white. It makes sense to decimate the chrominance signal. To represent the different sampling schemes, numbers are used. How many pixel values are sent from 4 original pixels for each Y, Cb, Cr component. 4:2:2 represent 4 Y samples and 2 Cb, 2 Cr samples. 4:1:1 represent 4 Y samples and 1 Cb, 1 Cr samples 4:2:0 represent 4 Y samples and 2 avg. chroma samples In practice, 1 Cb and 1 Cr samples from mid point. 19/31

20 20/31 Digital Video

21 Digital Video CCIR Standards for Digital Video Consultative Committee for International Radio CCIR-601 for component digital video (ITU-R-601) Adopted by DV video NTSC version 525 lines with 858 pixels (720 pixels visible) 4:2:2 format Each pixel represented with 2 bytes (1 for Y, 1 for Cb or Cr alternating) 525 x 858 x 30 x 2 byte ~ 216 Mbps without audio Use blanking for carrying audio, translation, error-correction, etc. 4:3 with interlacing 21/31

22 Digital Video CIF / QCIF Common Intermediate Format by CCITT (International Telegraph and Telephone Consultative Committee), Now ITU-T and ITU-R (International Telecommunication Union) VHS quality with less bitrate. Progressive scan Divisible by 8 and convenient for H.261 / H.263 video coding Compromise between NTSC and PAL Using NTSC frame rate and Half of the active lines in PAL 22/31

23 Digital Video Comparison of Digital Video Specification CCIR /60 NTSC CCIR /50 PAL/SECAM CIF QCIF Luminance Resolution Chrominance Resolution Color Subsampling 720x x x x x x x144 88x72 4:2:2 4:2:2 4:2:0 4:2:0 Aspect Ratio 4:3 4:3 4:3 4:3 Fields/Sec Interlaced Yes Yes No No 23/31

24 Digital Video High Definition TV (HDTV) Started NOT to provide better definition in each unit area, but to increase the visual field especially in its width. (Experience with wide screen movies) First generation developed by Sony and NHK in 1970s. Successful broadcast of 1984 LA Olympics in Japan. MUSE (Multiple sub-nyquist Sampling Encoding) was an improved NHK HDTV w/ hybrid analog / digital technologies scan lines with interlaced 16:9 aspect ratio. Used satellite with 24 MHz. Uncompressed HDTV demand more than 20 MHz bandwidth. Not appropriate for current 6 MHz / 8 MHz channels. Needs compression 24/31

25 Digital Video High Definition TV (HDTV) History 1987, FCC decided HDTV standards must be compatible with the existing NTSC standard and be confined to the existing VHF and UHF bands. 1990, FCC announced preference for a full-resolution HDTV and decided that HDTV would be simultaneously broadcast with the existing NTSC TV to gradually replace it. 1993, FCC decided to go full-digital for digital HDTV, and GI / MIT / Zenith / AT & T vs. Thomson / Philips / Sarnoff Formed ATSC (Advanced Television Systems Committee) Standard for TV broadcasting of HDTV. 1995, U.S. FCC Advisory Committee on Advanced Television Service recommended that ATSC DTV standard be adopted. MPEG-2 for Video, AC-3 for Audio (5.1 Channel) is chosen. 25/31

26 Digital Video I: interlaced scan, P: progressive scan # of Active pixels per line Number of Active lines Aspect Ratio Picture rate :9 60I, 30P, 24P :9 60I, 30P, 24P :9 & 4:3 60I, 60P, 30P, 24P :3 60I, 60P, 30P, 24P 26/31

27 Digital Video HDTV has wider aspect ratio. (16:9) Supports progressive scan, reducing serrated edges to moving edges and flickers along horizontal lines. In USA, FCC planed to fully replace by 2006 => Done on (all VHF transmissions) No more analog TV transmission. SDTV: NTSC quality or higher. EDTV: 480 active lines or higher HDTV: 720 active lines or higher In Korea, started to terminate analog TV on selected regions gradually starting from Mid 2010 to Jeju island in 2011, and terminated analog TV on nation wide. 27/31

28 Types of Video Interfaces - Analog Component Video Higher-End Video System using three separate video signals for the R, G, B image planes. Use separate physical connection for each signal. Can use YIQ or YUV. Computer systems w/ RGB signals use component video. Need more bandwidth and good synchronization of the three components. 28/31

29 Types of Video Interfaces - Analog Composite Video Color (chrominance) and Intensity (luminance) signals are mixed into a single carrier wave. Chrominance as a composite of two color components (I and Q / U and V) Used by broadcast color TVs to keep compatibility with black-and-white TVs. In NTSC, I and Q are combined into a chroma signal and a color subcarrier puts the chroma signal at the higher end of the channel shared with the luminance signal. Usually connected with BNC or RCA plug. Interference between the luminance and chrominance signals is inevitable. 29/31

30 Types of Video Interfaces - Analog S-video Stands for Separate Video or Super Video. Uses two separate connections, one for luminanace and one for composite chrominance signal. Enhances video quality by avoiding crosstalk/interference between color and gray-scale information. Sends less color detail compared to the gray-scale information. 30/31

31 Types of Video Interfaces - Analog VGA Video Graphics Array Introduced by IBM in 1987 Started from 640x480 using 15pin D-sub Extended to support up to 2048x1536 at 85Hz (388MHz Bandwidth) Based on RGBHV (red, green, blue, horizontal sync, vertical sync) 31/31

32 32/31 Types of Video Interfaces - Digital DVI Digital Visual Interface Developed to transfer digital video signals from video card to monitor Carries uncompressed video supporting DVI-D(digital only), DVI- A(Analog only), DVI-I(digital and analog) mode Based on PanelLink (high-speed serial link) Video card can read the display s capability by reading monitor s EDID block EDID: Extended display identification data Contains identification, color characteristics (including gamma level), supported video modes Single link supports Max 165 MHz pixel clock freq megapixels at 60Hz refresh 16:9 aspect ratio, 1920x1080 at 60Hz 2560x1600 at 60 Hz with dual link

33 Types of Video Interfaces - Digital HDMI High-Definition Multimedia Interface Backward compatible with DVI Popular in consumer electronics Defines protocols, signals, electrical interfaces, and mechanical req. Does not carry Analog signal Support RGB, YCbCr at 4:4:4 or 4:2:2 Support Digital Audio HDMI 1.0 supports 165 MHz (1080p at 60Hz) HDMI 1.3 supports 340 MHz (WQXGA 2560x1600) HDMI 2.0 released in 2013 supports 4K resolution at 60fps 33/31

34 3D Video - 거리 (Depth) 의인식 Stereopsis ( 양안시 / 입체시 ) Accommodation of the eye ( 원근조절 ) Occlusion of one object by another Subtended visual angle of an object of known size Linear perspective (convergence of parallel edges) Vertical position (objects higher in the scene generally tend to be perceived as further away) Haze, desaturation, and a shift to bluishness Change in size of textured pattern detail 원근법?

35 Stereoscopic Imaging 의원리 양안시차 ( 兩眼示差 ) 두눈의위치차이 ( 약 6.5cm) 로인하여발생하는각눈에입력되는영상의차이 초점 초점

36 Stereoscopic Imaging 의원리 입체감 / 거리의인식

37 Stereoscopic Image 좌안과우안에입력되는영상이조금씩다르도록하여입체감을느끼도록한다. 일반적으로두장의영상을사용

38 Stereoscopic 영상의취득 좌안용영상과우안용영상의취득 Stereoscopic Camera 의사용

39 Stereoscopic Image

40 3D 디스플레이방식 Side-by-Side 방식 안경방식디스플레이 적청안경방식 ( 애너글리프 (Anaglyph) 3D) 편광안경방식 ( 패시브글라스 ) 셔터안경방식 ( 액티브글라스 )

41 3D 디스플레이방식 적청안경방식 (Anaglyph 3D) 각눈을위한이미지를다른색을가진필터 ( 일반적으로보색관계에있는색, 적 / 청 ) 를이용하여만든다. 애너글리프 3D 영상은따라서각눈을위한영상, 즉두개의다른색으로필터된컬러영상을포함한다. 간단하게애너글리프영상을만드는법 좌안영상은청 / 녹색을제거하여만들며, 우안영상은적색을제거하여만든다. 두개의영상을 Main subject 가근사하게겹치도록만든다

42 3D 디스플레이방식 적청안경방식 (Anaglyph 3D) 3D 의인식 In a red-cyan anaglyph, the eye viewing through the red filter sees red within the anaglyph as "white", and the cyan within the anaglyph as "black". The eye viewing through the cyan filter perceives the opposite. Actual black or white in the anaglyph display, being void of color, are perceived the same by each eye. The brain blends together the red and cyan channelled images as in regular viewing but only green and blue are perceived. Red is not perceived because red equates with white through red gel and is black through cyan gel. However green and blue are perceived through cyan gel

43 3D 디스플레이방식 Color Code 3D (A type of Anaglyph 3D) stereo viewing system deployed in the 2000s that uses amber and blue filters nearly full color viewing (particularly within the RG color space) The blue filter is centered around 450 nm and the amber filter lets in light at wavelengths at above 500 nm

44 3D 디스플레이방식 Color Code 3D (A type of Anaglyph 3D) 2009 Super Bowl for SoBe, Monsters vs. Aliens animated movie and an advertisement for the Chuck television series in which the full episode the following night used the format. Time Inc. used Color Code 3-D in five of their magazines (Time, People, Sports Illustrated, Entertainment Weekly and Fortune) to display 3-D images when they published a series of articles about the new "3-D revolution" in April

45 Polarizer ( 편광기 / 편광필터 ) Optical filter that passes light of a specific polarization and blocks waves of other polarizations. There are linear and circular polarizers

46 3D 디스플레이방식 편광안경방식 ( 패시브글라스 ) 빛의편향성을이용하여각눈에도달하는빛을제어하여 3D 이미지로느끼도록하여주는방식 To present a stereoscopic motion picture, two images are projected superimposed onto the same screen through different polarizing filters. The viewer wears low-cost eyeglasses which contain a pair of different polarizing filters. As each filter passes only that light which is similarly polarized and blocks the light polarized in the opposite direction, each eye sees a different image. This is used to produce a three-dimensional effect by projecting the same scene into both eyes, but depicted from slightly different perspectives

47 3D 디스플레이방식 편광안경방식 ( 패시브글라스 ) 디스플레이패널의 FPR (Film-type Patterned Retarder) 필름과편광안경의 correlation 을이용하여좌안과우안에각각다른영상을동시에보여준다. Linearly polarized glasses Orthogonal polarizing filters (Usually at 45 and 135 degrees) Linearly polarized glasses require the viewer to keep his head level, as tilting of the viewing filters will cause the images of the left and right channels to bleed over to the opposite channel

48 3D 디스플레이방식 편광안경방식 ( 패시브글라스 ) Circularly polarized glasses eyeglasses which contain a pair of analyzing filters (circular polarizers mounted in reverse) of opposite handedness. Light that is left-circularly polarized is blocked by the right-handed analyzer, while right-circularly polarized light is extinguished by the left-handed analyzer. The result is similar to that of stereoscopic viewing using linearly polarized glasses, except the viewer can tilt his or her head and still maintain left/right separation. LG 전자 3D TV

49 3D 디스플레이방식 셔터안경방식 ( 액티브글라스 ) Shows left and right image alternately. When TV displays left eye image, the glasses block the right eye by closing the shutter of right glass lens. The display alternately displays different perspectives for each eye, using a technique called alternate-frame sequencing, which achieves the desired effect of each eye seeing only the image intended for it. 삼성전자 3D TV 영상처리프로그래밍 By Visual C++ 한빛미디어

50 50/31 Q & A