Final Report Multimedia Processing Term project on ERROR CONCEALMENT TECHNIQUES IN H.264 Spring 2016 Under Dr. K. R. Rao by Moiz Mustafa Zaveri (1001115920) moiz.mustafazaveri@mavs.uta.edu 1
Acknowledgement I would like to thank Dr. K.R. Rao (Department of Electrical Engineering at the University of Texas at Arlington) for his guidance and support at various stages of this project. His insight has been invaluable and his knowledge, resourceful. I would also like to thank the graduate teaching assistant Tuan Minh Pho for his help, insight and encouragement throughout the implementation of this project. 2
Contents 1. Problem Statement 2. Project Objective 3. The H.264 Standard 4. Sequence Characterization 5. Error Characteristics 6. Error Concealment Techniques 7. Quality Metrics 8. Generation of Errors 9. Simulation 10. Results of Frame Copy algorithm 11. Conclusions 12. Future Scope 13. References 3
Acronyms AVC AVS DSL Advanced Video Coding Audio Video Standard Digital Subscriber Line HVS Human Visual System IEC ISO ITU JM International Electrotechnical Commission International Organization for Standardization International Telecommunication Union Joint Model LAN Local Area Network MMS Multimedia Messaging Service MPEG Moving Picture Experts Group PSNR Peak signal to noise ratio SAD Sum of absolute differences SSIM Structural similarity index metric VCEG Video Coding Experts Group 4
Problem Statement Video transmission errors are errors in the video sequence that the decoder cannot decode properly. In real-time applications, no retransmission can be used, therefore the missing parts of the video have to be concealed. To conceal these errors, spatial and temporal correlations of the video sequence can be utilized. As H.264 [3] employs predictive coding, this kind of corruption spreads spatio-temporally to the current and consecutive frames. Project Objective To implement both the spatial domain and temporal domain categories of error concealment techniques in H.264 [3] with the application of the Joint Model (JM 19.0) reference software [3] and use metrics like the peak signal to noise ratio (PSNR) and mean squared error (MSE), blockiness [1], blurriness [1] and structural similarity index metric (SSIM) [4] in order to compare and evaluate the quality of reconstruction. The H.264 standard H.264/AVC [10], is an open licensed standard, which was developed as a result of the collaboration between the ISO/IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG). It is one of the most efficient video compression techniques available today. Some of its major applications include video broadcasting, video on demand, MMS over various platforms like DSL, Ethernet, LAN, wireless and mobile networks, etc. The H.264 encoder/decoder block diagram is shown in fig.1. Figure 1: H.264 encoder/decoder block diagram [1] 5
Sequence characterization We can extract two types of information from a video sequence: spatial and temporal, depending on which characteristics we are looking at. Temporal information Movement characteristic It is easier to conceal linear movements in one direction because we can predict pictures from previous frames (the scene is almost the same). If there are movements in many directions, finding a part of a previous frame to fit, is going to be more difficult. An example of this can be screen cuts. In fig.2, five frames of three video sequences (football match, village panorama and music video clip) are seen. In the music video sequence we have two scene cuts in the same number of frames as the village sequence, where we have a smooth movement in one direction. Obviously, it will be easier to conceal the village sequence. Speed characteristic The slower the movement of the camera, the easier it will be to conceal an error. We can see an example of two different video speeds if we compare village sequence with football sequence given in fig.2. Figure 2: Movement and speed [2] 6
Spatial information Smoothness of the neighborhood The smoothness of the neighborhood of the erroneous macroblock will determine the difficulty of the spatial concealment. In fig.3, there are three different cases in which the lost macroblock has to be concealed using the neighboring blocks. In the first one, it is going to be easy to reconstruct the lost macroblock because the neighborhood is very uniform (smooth), with almost no difference between the neighboring macroblocks. In the second situation, it is going to be a little bit more difficult; we have to look for the edges, and then, recover the wire line. The third case is an example where the neighbors cannot help us to recover the macroblock because they do not give any information about the lost part (in this case, the eye). Error Characteristics Lost information Size and form of the lost region I/P frame Figure 3: Smoothness of the neighborhood [2] - If the error is situated in an I frame, it is going to affect more critically the sequence because it will affect all the frames until the next I frame and I frames do not have any other reference but themselves. - If the error is situated in a P frame it will affect the rest of the frames until the next I frame but we still have the previous I frame as a reference. 7
Error Concealment Techniques The main task of error concealment is to replace missing parts of the video content by previously decoded parts of the video sequence in order to eliminate or reduce the visual effects of bit stream error. Error concealment exploits the spatial and temporal correlations between the neighboring image parts within the same frame or from the past and future frames. The various error concealment methods can be divided into two categories: error concealment methods in the spatial domain and error concealment methods in the time domain. Spatial domain error concealment utilizes information from the spatial smoothness nature of the video image. Each missing pixel of the corrupted image part is interpolated from the intact surroundings pixels. Weighted averaging is an example of a spatial domain error concealment method. Temporal domain error concealment utilizes the temporal smoothness between adjacent frames within the video sequence. The simplest implementation of this method is replacing the missing image part with the spatially corresponding part inside a previously decoded frame, which has maximum correlation with the affected frame. Examples of temporal domain error concealment methods include the copy-paste algorithm, the boundary matching algorithm and the block matching algorithm. Spatial Error Concealment All error concealment methods in spatial domain are based on the same idea which says that the pixel values within the damaged macroblocks can be recovered by a specified combination of the pixels surrounding the damaged macroblocks. In this technique, the interpixel difference between adjacent pixels for an image is determined. The interpixel difference is defined as the average of the absolute difference between a pixel and its four surrounding pixels. This property is used to perform error concealment. The first step in implementing spatial based error concealment is to interpolate the pixel values within the damaged macroblock from four next pixels in its four 1-pixel wide boundaries. This method is known as weighted averaging, because the missing pixel values can be recovered by calculating the average pixel values from the four pixels in the four 1-pixel wide boundaries of the damaged macroblock weighted by the distance between the missing pixel and the four macroblocks boundaries (upper, down, left and right). This is shown in fig.4. 8
Figure 4: Weighted averaging algorithm [2] The formula used for weighted averaging is as follows [2]: Temporal Error Concealment It is easier to conceal linear movements in one direction because pictures can be predicted from previous frames (the scene is almost the same). If there are movements in many directions or scene cuts, finding a part of previous frame that is similar is more difficult, or even impossible. Frame copy algorithm It is easier to conceal linear movements in one direction because pictures can be predicted from previous frames (the scene is almost the same). If there are movements in many directions or 9
scene cuts, finding a part of previous frame that is similar is more difficult, or even impossible. Replaces the missing image part in the current frame with the spatially corresponding part inside a previously decoded frame, which has maximum correlation with the affected frame. It is easier to conceal linear movements in one direction because pictures can be predicted from previous frames (the scene is almost the same). When (i,j) values are missing from the current frame n, they can be concealed using the (i,j) values from the previous n-1 frame. This is shown in fig.5. Figure 5: Copying (i,j) pixel values from previous n-1 frame into current n frame [1] If there are movements in many directions or scene cuts, finding a part of previous frame that is similar is going to be more difficult, or even impossible. The slower is the movement of the camera, the easier will be to conceal an error. This method only performs well for low motion sequences but its advantage lies in its low complexity. Boundary matching algorithm Let B be the area corresponding to a one pixel wide boundary of a missing block in the nth frame Fn. Motion vectors of the missing block as well as those of its neighbors are unknown. The coordinates [x, y ] of the best match to B within the search area A in the previous frame F n 1 have to be found. This is shown in fig.6. The equation used is as follows: [1] (2) The sum of absolute differences (SAD) is chosen as a similarity metric for its low computational complexity. The size of B depends on the number of correctly received neighbors M, boundaries of which are used for matching. 10
Figure 6: Boundary matching algorithm to find best match for area B in search area A of the previous frame [1] Block matching algorithm Better results can be obtained by looking for the best match for the correctly received MBD : D {T, B, L, R} which are the top, bottom, left or right side blocks of the missing macro block. The equation used is as follows: [1] (3) where AD represents the search area for the best match of MBD, with its center spatially corresponding to the start of the missing MB. The final position of the best match is given by an average over the positions of the best matches found for the neighboring blocks, computed as follows: [1] x = 1 E y = 1 E Where E is the number of correctly received neighbors. x D D (4) y D D (5) An MB sized area starting at the position [x, y ] in Fn 1 (previous frame) is used to conceal the damaged MB in Fn (current frame). To reduce the necessary number of operations, only parts of the neighboring MBs can be used for the MV search. Motion Vector Interpolation algorithm A motion vector of a 4x4 block can be estimated by interpolating its value from the motion vectors in the surrounding macroblocks, as shown in fig.8. The distance between the 4x4-block 11
to be found and the surrounding 4x4 blocks can be used as a weighting factor. This is shown in fig.7. Figure 7: Weighted averaging in Motion Vector Interpolation [1] Quality Metrics Figure 8: Motion Vector Interpolation [1] An objective image quality metric can play a variety of roles in image processing applications. First, it can be used to dynamically monitor and adjust image quality. For example, a network digital video server can examine the quality of video being transmitted in order to control and allocate streaming resources. Second, it can be used to optimize algorithms and parameter settings of image processing systems. Third, it can be used to benchmark image processing systems and algorithms. 12
In this project the following quality metrics are used: i. Peak Signal to Noise Ratio (PSNR) ii. iii. Distortion Artifacts (blockiness and blurriness) Structural Similarity Index Metric (SSIM) Peak Signal to Noise ratio (PSNR) In scientific literature it is common to evaluate the quality of reconstruction of a frame F by analyzing its peak signal to noise ratio (PSNR). There are different ways of calculating PSNR. One is frame-by-frame and the other is the overall average. The PSNR for an 8 bit PCM (0-255 levels) is calculated using: [1] (255) PSNR = 10. log 2 10 [db] (6) MSE Where MSE is the mean square error, given by: [1] MSE = 1 WX W X i=1 j=1[f(i, j) F 0 (i, j)] 2 (7) Where, W X is the size of the frame, F 0 is the original frame and F is the current frame. The YUV-PSNR for the 4:2:0 format is calculated using: [1] PSNR YUV = [6(PSNR) Y + (PSNR) U + (PSNR) V ]/8 (8) Where, (PSNR) Y is the PSNR of the Y component, (PSNR) U is the PSNR of the U component and (PSNR) V is the PSNR of the V component Distortion Artifacts (blockiness and bluriness) Here measurement of distortion artifacts like blockiness and blurring is done. Blockiness is defined as the distortion of the image characterized by the appearance of an underlying block encoding structure [1]. This metric was created to measure the visual effect of blocking. If the value of the metric for first picture is greater than the value for the second picture, it means that first picture has more blockiness, than the second picture. This is seen in fig.9 where the second picture has more underlying blockiness than the first picture. On the other hand, blurriness is 13
defined as a global distortion over the entire image, characterized by reduced sharpness of edges and spatial detail [1]. This metric compares the power of blurring of two images. If the value of the metric for first picture is greater than the value for the second picture, it means that second picture is more blurred, than first. This is seen in fig.10, where the back wing of the butterfly in the second picture is blurred in comparison to the first picture. Figure 9: (a) Orignal image (b) Blockiness in original image [1] Figure 10: (a) Orignal image (b) Blurriness in original image [1] 14
Structural Similarity Index Metric (SSIM) [4] The main function of the human visual system (HVS) is to extract structural information from the viewing field, and HVS is highly adapted for this purpose. Therefore, a measurement of structural information loss can provide a good approximation to perceived image distortion. SSIM compares local patterns of pixel intensities that have been normalized for luminance and contrast. The luminance of the surface of an object being observed is the product of the illumination and the reflectance, but the structures of the objects in the scene are independent of the illumination. Consequently, to explore the structural information in an image, the influence of illumination must be separated. The block diagram for measuring the SSIM is given in fig.11. Figure 11: SSIM measurement [1] Let x and y be two image patches extracted from the same spatial location of two images being 2 2 2 compared. Let μ x and μ y be their means and σ x and σ y be their variances. Also, let σ xy be the variance of x and y. The luminance, contrast and structure comparison are given by: [1] (9) (10) 15
(11) Where C 1, C 2 and C 3 are all constants given by: [1] (12) L is the dynamic range of the pixel values (L = 255 for 8 bits/pixel gray scale images), and K 1 1 and K 2 1 are scalar constants. The general SSIM can be calculated as follows: [1] (13) Where α, β and γ are parameters which define the relative importance of the three components. Generation of errors This is done by using the rtp_loss file in the software solution. This program can be used to simulate frame loss on RTP files. The C random number generator is used for selecting the frames to be lost. The random number generator is not initialized before usage. Therefore subsequent runs of the tool will create identical loss pattern. The number of frames lost depends on the loss percent that the user provides. Example: rtp_loss.exe infile outfile losspercent <keep_leading_packets> where <keep_leading_packets> is an optional parameter that specifies the number of RTP frames that are kept at the beginning of the file. Figs. 12 and 13 show a single frame of Akiyo and Foreman sequences with and without frame loss. 16
Figure 12: (a) Original Akiyo frame (b) Akiyo frame with error Figure 13: (a) Original Foreman frame (b) Foreman frame with error Simulation The simulation steps for this project are quite straightforward. First the JM 19.0 reference software for H.264/AVC [3] has to be downloaded and run in Microsoft Visual Studio. Next the encoder and decoder configuration files have to be modifies to meet the requirements and to accept the desired inputs. encoder.cfg InputFile = "foreman_qcif.yuv" # Input sequence 17
FramesToBeEncoded = 150 # Number of frames to be coded FrameRate = 30.0 # Frame rate per second OutputFile = "enout_150.264" # Bitstream QPISlice = 28 # Quant. param for I Slices (0-51) QPPSlice = 28 # Quant. param for P Slices (0-51) OutFileMode = 1 # Output file mode, 0:Annex B, 1:RTP decoder.cfg InputFile OutputFile = "lossout_150.264" # H.264/AVC coded bitstream = "decout_150.yuv" # Output file, YUV/RGB FileFormat = 1 # NAL mode (0=Annex B, 1: RTP packets) ConcealMode Copy) =1 #ErrConcealment(0:Off,1:Frame Copy,2:Motion After the configuration files have been modified, we build the software solution in Microsoft Visual Studio. This creates four executable files (lencod.exe, ldecod.exe, rtp_loss.exe and rtpdump.exe). First we run the lencod.exe file in command prompt to obtain an encoded output.264 file. On completing execution of lencod.exe the command window outputs the measure of various parameters as seen in fig.14. Next the rtp_loss.exe file is run in command prompt to introduce a certain percentage of loss in the encoded output and obtain a lossy.264 file. Finally, the ldecod.exe file is executed to obtain the decoded output yuv file in which one of the two error concealment algorithms have been employed to conceal the frame losses. The command window output of ldecod.exe is seen in fig.15. 18
Figure 14: Encoder simulation output Figure 15: Decoder simulation output Test Sequences The test sequences that have been used include: akiyo_qcif.yuv, foreman_qcif.yuv, stefan_qcif.yuv [5]. Each of these sequences has a total of 300 frames out of which 150 frames are encoded. The dimensions of the frames are 176x144. The frames are encoded with the quantization parameter set to 28 at a rate of 30 frames/second. Two common frame formats are: Common Intermediate Format (CIF) and Quadrature Common Intermediate Format (QCIF). These determine the resolution of the frame. The resolution of CIF 19
is 352x288, as seen in fig. 16, and the resolution of QCIF is 1/4 of CIF, which is 176x144, seen in fig.17. In this project QCIF frame sequences are used with a format of 4:2:0. Figure 16: Common Intermediate Format (CIF) [7] Figure 17: Quadrature Common Intermediate Format (QCIF) [7] Results of Frame Copy algorithm In this project, each of the three video sequences: Akiyo.yuv, Foreman.yuv and Stefan.yuv [5] have been made erroneous with loss of frames ranging from 1% to 5% of the total number of encoded frames. The errors in the resultant lossy sequences have then been concealed using the Frame Copy algorithm. The resultant video sequences have been analyzed using metrics like: PSNR, MSE, SSIM [4], blockiness and blurriness and the results have been tabulated and plotted. 20
Results for Akiyo.yuv Figure 18: Original Akiyo sequence frame Figure 19: Erroneous Akiyo sequence frame 21
Figure 20: Error concealed Akiyo sequence frame The error concealment seen in the decoded output sequence of the erroneous Akiyo sequence is not perfect, as seen in the story told by figs. 18-20. The frame copy algorithm has simply copied the pixels of the same location from the previous frame. With an increase in the number of frames lost, it is seen that the PSNR (in table 1 and fig.21), SSIM [4] (in table 2 and fig.22) and blurriness (in table 5 and fig.25) of the output sequence reduce while the MSE (in table 3 and fig.23) and blockiness (in table 4 and fig.24) increase. Error Percentage (%) Average PSNR of concealed result (db) 1 30.243 2 30.240 3 30.179 4 30.09 5 29.975 Table 1: Average PSNR (db) of the error concealed output for different error percentages 22
Figure 21: Average PSNR (db) vs error percentage for the Akiyo sequence Error Percentage (%) Average SSIM of concealed result 1 0.92751 2 0.92747 3 0.92704 4 0.92629 5 0.92527 Table 2: Average SSIM values of the error concealed output for different error percentages 23
Figure 22: Average SSIM vs error percentage for the Akiyo sequence Error Percentage (%) Average MSE of concealed result 1 61.48661 2 61.52298 3 62.39262 4 63.68896 5 65.20619 Table 3: Average MSE values of the error concealed output for different error percentages 24
Figure 23: Average MSE vs error percentage for the Akiyo sequence Error Percentage (%) Original sequence Blockiness Measure Concealed sequence Blockiness Measure 1 11.81973 13.79751 2 11.81973 13.80202 3 11.81973 13.81160 4 11.81973 13.81890 5 11.81973 13.82310 Table 4: Average blockiness of the original sequence and the error concealed output for different error percentages 25
Figure 24: Average blockiness vs error percentage for the Akiyo sequence Error Percentage (%) Original sequence Blurriness Measure Concealed sequence Blurriness Measure 1 16.32378 14.92913 2 16.32378 14.92850 3 16.32378 14.91929 4 16.32378 14.91371 5 16.32378 14.90696 Table 5: Average blurriness of the original sequence and the error concealed output for different error percentages 26
Figure 25: Average blurriness vs error percentage for the Akiyo sequence Results for Foreman.yuv Figure 26: Original Foreman sequence frame 27
Figure 27: Erroneous Foreman sequence frame Figure 28: Error concealed Foreman sequence frame The error concealment seen in the decoded output sequence of the erroneous Foreman sequence is not perfect, as seen in the story told by figs.26-28. The frame copy algorithm has simply copied the pixels of the same location from the previous frame. With an increase in the number of frames lost, it is seen that the PSNR (in table 6 and fig.29), SSIM [4] (in table 7 and fig.30) and blurriness (in table 10 and fig.33) of the output sequence reduce while the MSE (in table 8 and fig.31) and blockiness (in table 9 and fig.32) increase. 28
Error Percentage (%) Average PSNR of concealed result (db) 1 17.25731 2 17.25303 3 17.22355 4 17.19582 5 17.16782 Table 6: Average PSNR (db) of the error concealed output for different error percentages Figure 29: Average PSNR (db) vs error percentage for the Foreman sequence 29
Error Percentage (%) Average SSIM of concealed result 1 0.54698 2 0.54622 3 0.54434 4 0.54222 5 0.53907 Table 7: Average SSIM of the error concealed output for different error percentages Figure 30: Average SSIM vs error percentage for the Foreman sequence 30
Error Percentage (%) Average MSE of concealed result 1 1222.65454 2 1223.86035 3 1232.19348 4 1240.08655 5 1247.15624 Table 8: Average MSE of the error concealed output for different error percentages Figure 31: Average MSE vs error percentage for the Foreman sequence 31
Error Percentage (%) Original sequence Blockiness Measure Concealed sequence Blockiness Measure 1 6.66808 15.81962 2 6.66808 15.83485 3 6.66808 15.87380 4 6.66808 15.95529 5 6.66808 16.11628 Table 9: Average blockiness of the error concealed output for different error percentages Figure 32: Average blockiness vs error percentage for the Foreman sequence 32
Error Percentage (%) Original sequence Blurriness Measure Concealed sequence Blurriness Measure 1 25.58911 24.63489 2 25.58911 24.63258 3 25.58911 24.62816 4 25.58911 24.62291 5 25.58911 24.59365 Table 10: Average blurriness of the error concealed output for different error percentages Figure 33: Average blurriness vs error percentage for the Foreman sequence 33
Results for Stefan.yuv For the Stefan sequence, with an increase in the number of frames lost, it is seen that the PSNR (in table 11 and fig.34), SSIM [4] (in table 12 and fig.35) and blurriness (in table 15 and fig.38) of the output sequence reduce while the MSE (in table 13 and fig.36) and blockiness (in table 14 and fig.37) increase. Error Percentage (%) Average PSNR of concealed result (db) 1 15.74143 2 15.74008 3 15.72960 4 15.71948 5 15.71072 Table 11: Average PSNR (db) of the error concealed output for different error percentages Figure 34: Average PSNR (db) vs error percentage for the Stefan sequence 34
Error Percentage (%) Average SSIM of concealed result 1 0.42509 2 0.42466 3 0.42389 4 0.42273 5 0.42116 Table 12: Average SSIM of the error concealed output for different error percentages Figure 35: Average SSIM vs error percentage for the Stefan sequence 35
Error Percentage (%) Average MSE of concealed result 1 1733.18079 2 1733.72205 3 1737.91101 4 1741.96741 5 1745.87162 Table 13: Average MSE of the error concealed output for different error percentages Figure 36: Average MSE vs error percentage for the Stefan sequence 36
Error Percentage (%) Original sequence Blockiness Measure Concealed sequence Blockiness Measure 1 15.21208 14.98759 2 15.21208 15.01642 3 15.21208 15.03965 4 15.21208 15.06558 5 15.21208 15.12065 Table 14: Average blockiness of the error concealed output for different error percentages Figure 37: Average blockiness vs error percentage for the Stefan sequence 37
Error Percentage (%) Original sequence Blurriness Measure Concealed sequence Blurriness Measure 1 43.92272 42.76152 2 43.92272 42.75115 3 43.92272 42.73378 4 43.92272 42.70599 5 43.92272 42.65922 Table 15: Average blurriness of the error concealed output for different error percentages Figure 38: Average blurriness vs error percentage for the Stefan sequence 38
Conclusions From the results obtained from decoding and hence concealment of the erroneous sequences we can make the following conclusions: The PSNR reduces with an increase in the error percentage. The SSIM reduces with an increase in the error percentage. The MSE increases with an increase in the error percentage. As compared to the input, the average blockiness of the error concealed output is higher when there is an increase in the percentage of error. This means that the error concealed output suffers from more blockiness than the input. As compared to the input, the average blurriness of the error concealed output is lower when there is an increase in the percentage of error. This means that the error concealed output suffers from more blurriness than the input. Considering Akiyo sequence to have less movement, Foreman sequence to have medium movement and Stefan sequence to have high movement, we see that: The PSNR of the concealed output decreases with an increase in movement. (From the range of 30dB in Akiyo.yuv sequence to the range of 17dB in Foreman.yuv sequence to the range of 15dB in Stefan.yuv sequence) The SSIM of the concealed output decreases with an increase in movement. (From the range of 0.92 in Akiyo.yuv sequence to the range of 0.54 in Foreman.yuv sequence to the range of 0.42 in Stefan.yuv sequence) The MSE of the concealed output increases with an increase in movement. (From the range of 60 in Akiyo.yuv sequence to the range of 1220 in Foreman.yuv sequence to the range of 1730 in Stefan.yuv sequence) 39
Future Work The current JM reference software (19.0) [3], implements only two error concealment algorithms. They are - Frame Copy algorithm and Motion Vector Copy algorithm. These are basic spatial and temporal error concealment algorithms. There is a need for implementation of better and probably hybrid concealment algorithms. Also, the 19.0 JM software [3] has poor error resiliency. This is seen in the fact that it cannot decode sequences which have more than a 5% frame loss error. It is true that transmission error losses in wireless networks seldom exceed this 5% threshold, but there can be exceptional cases. 40
References: [1] I. C. Todoli, Performance of Error Concealment Methods for Wireless Video, Ph.D. thesis, Vienna University of Technology, 2007. [2] V.S. Kolkeri "Error Concealment Techniques in H.264/AVC, for Video Transmission over Wireless Networks", M.S. Thesis, Department of Electrical Engineering, University of Texas at Arlington, Dec. 2009 Online: http://www.uta.edu/faculty/krrao/dip/courses/ee5359/index_tem.html. [3] H.264/AVC Reference Software Download: Online: http://iphome.hhi.de/suehring/tml/download/ [4] Z. Wang, The SSIM index for image quality assessment, Online: http://www.cns.nyu.edu/zwang/files/research/ssim/. [5] Video Trace research group at ASU, YUV video sequences, Online: http://trace.eas.asu.edu/yuv/index.html. [6] MSU video quality measurement tool: Online: http://compression.ru/video/quality_measure/video_measurement_tool_en.html. [7] S. K. Kwon, A. Tamhankar and K.R. Rao, An overview of H.264 / MPEG-4 Part 10, J. Visual Communication and Image Representation, vol. 17, pp.186-216, Apr. 2006. [8] Y. Xu and Y. Zhou, H.264 Video Communication Based Refined Error Concealment Schemes, IEEE Transactions on Consumer Electronics, vol. 50, issue 4, pp. 1135 1141, Nov. 2004. [9] M. Wada, Selective Recovery of Video Packet Loss using Error Concealment, IEEE Journal on Selected Areas in Communication, vol. 7, issue 5, pp. 807-814, June 1989. [10] Y. Chen et al, An Error Concealment Algorithm for Entire Frame Loss in Video Transmission, IEEE Picture Coding Symposium, Dec. 2004. [11] S. K. Bandyopadhyay et al, An error concealment scheme for entire frame losses for H.264/AVC, IEEE Sarnoff Symposium, pp. 1-4, Mar. 2006. [12] H. Ha, C. Yim and Y. Y. Kim, Packet Loss Resilience using Unequal Forward Error Correction Assignment for Video Transmission over Communication Networks, ACM digital library on Computer Communications, vol. 30, pp. 3676-3689, Dec. 2007. [13] T. Wiegand et al, Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, no. 7, July 2003. 41
[14] Z. Wang et al, Image quality assessment: From error visibility to structural similarity, IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004. [15] D. Levine, W. Lynch and T. Le-Ngoc, Observations on Error Detection in H.264, 50th IEEE Midwest Symposium on Circuits and Systems, pp. 815-818, Aug. 2007. [16] B. Hrušovský, J. Mochná and S. Marchevský, Temporal-spatial Error Concealment Algorithm for Intra-Frames in H.264/AVC Coded Video, 20th International Conference Radioelektronika, pp. 1-4, Apr. 2010. [17] W. Kung, C. Kim and C. Kuo Spatial and Temporal Error Concealment Techniques for Video Transmission Over Noisy Channels, IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, issue 7, pp. 789-803, July 2006. 42