The Lecture Contains: Sampling of Video Signals Choice of sampling rates Sampling a Video in Two Dimensions: Progressive vs. Interlaced Scans file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_1.htm[12/31/2015 1:03:13 PM]
Sampling of Video Signals Introduction We consider specifically the sampling of 3-D video signals. The two spatial dimensions and the temporal dimension are asymmetric. This implies that they have different characteristics and that the visual sensitivities to spatial and temporal frequencies are different. This asymmetry has led to the development of many interesting techniques for video sampling. As video signals vary continuously in space and time, no cameras of recent times can capture the entire signal continuously in all 3-dimensions. For example, 1. Most motion picture cameras sample a scene in temporal direction and store the sequence of analog frames on film. 2. Most TV cameras capture a video sequence by sampling it in temporal and vertical directions respectively. The resulting signal is stored as a 1-D raster scan, which is a joined segments of color variations along successive horizontal scan lines. To obtain a full digital video, one can 1. Sample in 2-D the analog frames resulting from motion picture camera; or 2. Sample in 1-D along the raster scan resulting from TV camera; or 3. Acquire digital video frames directly from a digital camera by sampling a scene in 3-D; The different sampling schemes correspond to different sampling lattices. For designing video sampling system, two important requirements are: 1. Choice of necessary sampling rates for video 2. Choice of a suitable efficient sampling lattice under a given total data or sampling rate. file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_2.htm[12/31/2015 1:03:14 PM]
Choice of sampling rates For designing a video sampling system in either 1-D, 2-D or 3-D, the fundamental question is what the spatial and temporal sampling resolutions should be. This is governed by several factors: 1. The frequency content of the underlying signal, 2. The visual thresholds in terms of the spatial and temporal cut off frequencies; 3. The capture and device characteristics; 4. The affordable processing, storage and transmission cost. If one uses a cubic lattice for sampling, then based on sampling theorem, the sampling rate in each dimension should be at least twice the highest frequency along that direction. Given that the maximum frequency in the signal can vary significantly, the visual cut-off frequencies which are the highest spatial & temporal frequencies that can be observed by the human visual system (HVS), should be the deciding factor in determining the sampling rates for video. There is no need to accommodate frequency components beyond these values. Now we also know that visual sensitivity depends on the mean brightness of the display. For TV signals, which are very bright, the visual thresholds lead to a frame rate, and a spatial frequency of 30cpd. For a normal viewing distance of 3 times the screen height, a spatial frequency of 25 cpd translates to 500 lines/frame. To sample each line, the horizontal sampling interval should match the vertical interval, so that the resulting pixels are square Hence for a display of 500 lines and, number of pixels/line is. file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_3.htm[12/31/2015 1:03:14 PM]
These sampling rates, required by visual cut-off frequencies were beyond what practical technologies could offer at the time when analog TV systems were developed. To reduce the data rate and consequently the cost for video capture, transmission & display, interlaced scan was developed. This process trades off vertical resolution for increased temporal resolution for a given total data rate. As an example, in NTSC TV system, 60 fields are captured per sec, but each field contains only half the desired number of lines. The total data rate is the same as that with a progressive scan with 30fps & 480lpf. By interlacing the lines scanned in each field, it can produce the same quality as a progressive scan using 60fps and 480 lpf, if the underlying scene is stationary. For high motion scene with vertical line patterns, it can lead to the infamous interlacing artefact. The interlaced format is retained mainly for compatibility with analog TV system. The HDTV system enhances the visual impact with an IAR of 16:9 and sampling resolution of 60fps & 720lpf. Again for compatibility purpose, an interlaced format with 60 fields/s & 540 lines/field can also be used. For motion pictures, because of reduced visual sensitivity in a movie theatre, where ambient brightness is low, a frame rate of 24fps (progressive) is used. Actually the originals image captured at 24fps, when played back, a blade that rotates 3 times per frame is placed before the projection lens, so that effective frame rate is 72fps. This suppresses flicker artifacts that might be experienced. For computer display, e.g. SVGA display has a frame rate of 72fps & spatial resolution of pixels. This is to accommodate the very close viewing distance & high frequency content of displayed material (text & graphics). file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_4.htm[12/31/2015 1:03:14 PM]
Sampling a Video in Two Dimensions: Progressive vs. Interlaced Scans A video raster scan (progressive and interlaced scans) is actually a version of a 3-D video signal sampled in the temporal and vertical directions. In this section we attempt to gain some insight into the artifacts associated with these two sampling schemes, by analysing their aliasing patterns in the spectral domain. Ignoring the horizontal direction, we consider the video signal as a 2-D signal in the space spanned by the temporal and vertical directions. Let represent the field interval, and the line interval. The sampling lattices imployed by progressive and interlaced scans are shown in Fig.( ) and ( ). The basis vectors for generating each lattice are also shown. Using these basis vectors, we derive the generating matrices for the original and reciprocal lattices. Progressive Scan: Interlaced Scan: file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_5.htm[12/31/2015 1:03:14 PM]
(Figure 4) Comparison of progressive and interlaced scans (a) sampling lattice for PS (b) sampling lattice for IS (c) reciprocal lattice for PS (d) reciprocal lattice for IS. file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_6.htm[12/31/2015 1:03:14 PM]
Referring to Fig (4), we compare the original and reciprocal lattices of the two scans and observe the following characteristics: a. The sampling densities are the same for the two scans b. Along the vertical axis, the nearest alias occurs at for both the scans. This implies that in the absence of motion, the two sampling lattices have the same vertical resolving power. This is because although there are only half the number of lines in each field, when the image is stationary, the lines sampled at two different fields appear as if they are sampled at the same time. When there is motion in the scene, the vertical resolution of interlaced scan is less than that of progressive scan. c. The two scans have different nearest aliases along the temporal frequency axis. In PS the first alias is at and for IS it is at. As a temporal frequency component larger than half the nearest temporal alias is likely to cause flicker rtifact, we observe that IS is less prone to flickering when the object has a flat or slowly varying pattern. d. The two scans have different nearest off-axis aliases. They are also referred to as mixed aliases. A frequency component close to mixed alias gives rise to interline flicker or line crawl for the progressive scan, the mixed alias occurs at, whereas for the interlaced scan, the mixed alias is at. As the mixed alias in case of interlaced scan is closest to origin, the interline flicker is more visible in interlaced scan. These are referred to as notorious interlacing artifacts. e. For a signal with symmetric support, interlaced scan is more efficient. The maximum radius of the signal spectrum that can be represented without aliasing is equal to with progressive scan and this is increased to for interlaced scan. Of course, it is important to note that this result is based on our way of equating the spatial and temporal frequencies. The above comparisons between the two scanning schemes are based on the assumption that total sampling rate is the same in both cases. The interlaced scan has advantages over the progressive scan. The quality can be improved further by deinterlacing the interlaced scan. This yields a progressive scan that has twice the total sampling rate. file:///d /...e%20(ganesh%20rana)/my%20course_ganesh%20rana/prof.%20sumana%20gupta/final%20dvsp/lecture16/16_7.htm[12/31/2015 1:03:15 PM]