A Study on AVS-M video standard

1 A Study on AVS-M video standard EE 5359 Sahana Devaraju University of Texas at Arlington Email:sahana.devaraju@mavs.uta.edu

2 Outline Introduction Data Structure of AVS-M AVS-M CODEC Profiles & Levels Major and Minor Tools of AVS-M Error Concealment & Resilience Conclusions Results

3 Introduction To Audio Video Standard for Mobile (AVS-M) AVS is a set of integrity standard system - system, video, audio and media copyright management. AVS-M is the seventh part of the video coding standard developed by AVS workgroup of China which aims for mobile systems and devices. In AVS-M, a Jiben Profile has been defined which has 9 different levels. AVS follows a layered structure for the data and this representation is seen in the coded bitstream. Sequence layer provides an entry point into the coded video. It consists of a set of mandatory and optional downloadable parameters.

4 Picture The Picture layer provides the coded representation of a video frame. It comprises of a header with mandatory and optional parameters and optionally with user data Three types of Pictures are defined by AVS: Intra Pictures (I-pictures) Predicted Pictures (P-pictures) Interpolated Pictures (B-pictures) 4:2:0 subsampling format is used in AVS-M. Motion vectors can exceed the boundaries of the reference picture

5 Picture AVS-M supports only I picture and P picture which can be seen in Figure 1. AVS-M supports only progressive video sequence. Therefore, one picture is one frame. P picture can have a maximum of two reference frames for forward prediction. I P P P P P P Figure 1: Picture Types in AVS Part 7 [2]

6 Slice Slices comprise of a series of MB s. Slices must not overlap, must be contiguous, must begin and terminate at the left and right edges of the Picture. A single slice can cover the entire Picture. Slices are independently coded no slice can refer to another slice during the decoding process. Figure 2: Slice Structure for AVS P7 [5]

7 Macroblocks and Blocks Picture is divided into Macroblocks. The upper-left sample of each MB should not exceed picture boundary. Macroblock partitioning is used for motion compensation. The number in each rectangle specifies the order of appearance of motion vectors. Figure 3: Macroblock Partitioning [2]

8 AVS-M CODEC Each input MB needs to be intra predicted or inter predicted. In an AVS-M encoder, S0 is used to select the right prediction method for current MB whereas in the decoder, the S0 is controlled by the MB type of current MB. The intra predictions are derived from the neighbouring pixels in left and top blocks. The unit size of intra prediction is 4 4 because of the 4 4 integer cosine transform (ICT) used by AVS- M. The inter predictions are derived from the decoded frames. AVS-M employs an adaptive variable length coding (VLC) coding technique.

9 AVS-M CODEC The reconstructed image is the sum of prediction and current reconstructed error image. AVS-M uses the deblocking filter in motion compensation loop. The deblocking process directly acts on the reconstructed reference first across vertical edges and then across horizontal edges.

10 AVS-M ENCODER Figure 4 : Block Diagram of AVS-M encoder [5]

11 AVS-M DECODER Figure 5 : Block Diagram of AVS-M Decoder [5]

12 PROFILES AND LEVELS AVS-M defines Jiben Profile. There are nine levels specified which are: 1.0 : up to QCIF and 64kbps 1.1 : up to QCIF and 128kbps 1.2 : up to CIF and 384kpbs 1.3 : up to CIF and 768kbps 2.0 : up to CIF and 2Mbps 2.1 : up to HHR and 4Mbps 2.2 : up to SD and 4Mbps 3.0 : up to SD and 6Mbps 3.1 : up to SD and 8Mbps

13 Major and Minor tools of AVS-M Network Abstraction Layer (NAL) Supplemental Enhancement Information (SEI) Transform 4x4 integer transform Quantization and scaling - scaling only in encoder Intra prediction 9 modes, simple 4x4 intra prediction and direct intra prediction Motion compensation 16x16/16x8/8x16/8x8/8x4/4x8/4x4 modes Quarter-pixel interpolation 8-tap horizontal interpolation filter and 4-tap vertical interpolation filter Simplified in-loop deblocking filter Entropy coding Error resilience

14 Network Abstraction Layer In AVS-M video compression, a compressed video bitstream is made up of Access Units (AUs), and each AU contains information for decoding a picture. An AU consists of a number of NAL units, some of which are optional. A NAL unit can be a sequence parameter set (SPS), a picture parameter set (PPS), an SEI, a picture header, or a slice_layer_rbsp (raw byte sequence payload) which consists of a slice_header followed by slice data

15 Transform 4x4 is the unit of transform, intra prediction and smallest motion compensation in AVS Part 7. The 4x4 transform used in AVS is AVS-M uses a prescaled integer transform (PIT) technology; all of the scale-related operations have been done in the encoder. The decoder does not need any scale operations. PIT is used in AVS Part 7 to reduce the complexity.

16 Quantization Quantization of the transform coefficients is performed with an adaptive linear quantizer. The step size of the quantizer can be varied to provide rate control. The transmitted step size quantization parameter is used directly for luminance coefficients and for chrominance coefficients it is modified on the upper end of its range. The quantization parameter varies from 0 to 63 in steps of one.

17 Intra Prediction Two types of intra prediction modes are adopted in AVS-P7, Intra_4x4 and Direct Intra Prediction (DIP). AVS-P7 s intra coding brings a significant complexity reduction and maintains a comparable performance. Intra_4x4 Each 4x4 block is predicted from spatially neighbouring samples. For each 4x4 block, one of nine prediction modes can be utilized to exploit spatial correlation including eight directional prediction modes (such as Down Left, Vertical, etc) and non-directional prediction mode (DC).

18 Intra Prediction The 16 samples of the 4x4 block which are labeled as a-p are predicted using prior decoded samples in adjacent block label as A-D, E-H and X. The up-right pixels used to predict are expanded by pixel sample D and the down-left pixels are expanded by H. Figure 6:Eight Directional Prediction modes of AVS P7 [3]

19 Intra Prediction One of the nine prediction modes shown below is used for spatial corellation. Figure 7:Nine Intra_4 4 Prediction Modes of AVS P7 [10]

20 Content-based Most Probable Intra Mode Decision A statistical model is used to determine the most probable intra mode of current block based on video characteristics and content correlation. A lookup table is used to predict the most probable intra mode decision of current block. Irrespective of whether Intra_4x4 or DIP is used, the most probable mode decision method is described as follows: Get the intra mode of up block and left block. If the up (or left) block is not available for intra mode prediction, the mode of up (or left) block is defined as -1. Use the up intra mode and left intra mode to find the most probable mode in the table.

21 Content-based Most Probable Intra Mode Decision If current MB is coded as Intra_4x4 mode, the intra prediction mode is coded as follows. If the best mode equals to most probable mode, 1 bit of flag is transmitted to each block to indicate the mode of current block is its most probable mode. If the best mode is not the most probable mode, the 1 bit flag is to indicate the mode of current block is not the most probably mode, and then a 3 bit mode information is transmitted. Thus mode information of each block can be presented in 1 bit or 4 bits.

22 Direct Intra Prediction When direct intra prediction is used, a new method is followed to code the intra prediction mode information. A rate-distortion based direct intra prediction mainly contains 5 steps. Step 1: All 16 4 4 blocks in a MB use their most probable modes to do Intra_4 4 prediction and calculate RDCost(DIP) of this MB. (1) RDCost(mode)=D(mode) + λ.r(mode) Step 2: Mode search of Intra_4 4, find the best intra prediction mode of each block, and calculate RDCost(Intra_4x4).

23 Direct Intra Prediction Step 3: Compare RDCost(DIP) and RDCost(Intra_4x4). If RDCost(DIP) is less than RDCost(Intra_4x4), DIP flag equals to 1 then go to step 4, else DIP flag equals to 0 go to step 5. Step 4: Encode the MB using DIP and finish encoding of this MB. Step 5: Encode the MB using ordinary Intra_4 x4 and finish encoding of this MB.

24 INTERFRAME PREDICTION AVS-M defines I picture and P picture. P pictures use forward motion compensated prediction. The maximum number of reference pictures used by a P picture is two. AVS-M also specifies nonreference P pictures. If the nal_ref_idc of a P picture is equal to 0, the P picture shall not be used as a reference picture. The nonreference P pictures can be used for temporal scalability. The reference pictures are identified by the reference picture number, which is 0 for IDR picture. The reference picture number of a non-idr reference picture is calculated as refnum= (2)

25 INTERFRAME PREDICTION After decoding current picture, if nal_ref_idc of current picture is not equal to 0, then current picture is marked as used for reference. If current picture is an IDR picture, all reference pictures except current picture shall be marked as unused for reference. Otherwise, if nal_unit_type of current picture is not equal to 0 and the total number of reference pictures excluding current picture is equal to num ref frames, the following applies: If num ref frames is 1, reference pictures excluding current picture in DBP shall be marked as unused for reference. If num ref frames is 2 and sliding window size is 2, the reference picture excluding current picture in DPB with smaller reference picture number shall be marked as unused for reference. Otherwise, if num ref frames is 2 and sliding window size is 1, the reference picture excluding current picture in DBP with larger reference picture number shall be marked as unused for reference.

26 INTERFRAME PREDICTION The size of motion compensation block can be 16 16, 16 8, 8 16, 8 8, 8 4, 4 8 or 4 4. If the half_pixel_mv_flag is equal to 1, the precision of motion vector is up to 1/2 pixel, otherwise the precision of motion vector is up to ¼ pixel. When half_pixel_mv_flag is not present in the bitstream, it shall be inferred to be 11. The interpolated values at half sample positions can be obtained using 8-tap filter F1 = ( 1,4, 12,41,41, 12,4, 1) and 4-tap filter F2 = ( 1,5,5, 1).

27 INTERFRAME PREDICTION The positions of integer, half and quarter pixel samples are depicted in Figure 8. Capital letters indicate integer sample positions, while small letters indicate half and quarter sample positions. Figure 8: The Position of Integer, Half and Quarter Pixel Samples [3]

28 Deblocking Filter AVS Part 7 makes use of a simplified deblocking filter, wherein boundary strength is decided at MB level. Filtering is applied to the boundaries of luma and chroma blocks except for the boundaries of picture or slice. Intra prediction MB usually has more and bigger residuals than that of inter prediction MB, which leads to very strong blocking artifacts at the same QP. A stronger filter is applied to intra predicted MB and a weak filter is applied to inter predicted MB. When QP is not very large, the distortion caused by quantization is relatively small, henceforth no filtering is required.

29 Deblocking Filter If the following three conditions hold good then the filtering process is applied otherwise the filtering process is bypassed. p 0 -q 0 < α (IndexA) p 1 -p 0 < β (IndexB) q 1 -q 0 < β (IndexB) where α and β can be calculated by IndexA, IndexB. p 1, p 0, q 1 and q 0 are samples across every sample-level boundary. Figure 9: Horizontal or Vertical Edge of 4 4 Block

30 Entropy coding Entropy coding, involves mapping from a video signal after prediction and transforming to a variable length coded bitstream. AVS-M uses Exp-Golomb code, as shown in the table below to encode syntax elements such as quantized coefficients, macroblock coding type, and motion vectors. Eighteen coding tables are used in quantized coefficients encoding. The encoder uses the run and the absolute value of the current coefficient to select the table.

31 Entropy coding Figure 10:Kth Order Golomb Code [5]

32 Context Based Adaptive 2 Dimensional Variable length Coding In AVS an efficient context based adaptive 2D variable length coding is designed for coding transform coefficients in a 4 4 block. The transform coefficients are mapped into one dimensional (level, run) sequence by the reverse zigzag scan. It employs 2D joint VLC to remove the redundancy between the levels and runs in transform coefficients block. It employs multiple conditionally-trained 2D-VLC tables to better match different (level, run) s probability distributions at different coding phases by automatic table switching. It makes use of an improved table switching method and an improved escape coding method.

33 Context Based Adaptive 2 Dimensional Variable length Coding Also, it employs a new Coded Block Pattern (CBP), CBP_4 4 to be compatible with the 4 4 transform better. The transform block size in AVS is 4 4, so a new 4-bit syntax element CBP_4 4 is introduced.

34 Error Concealment To deal with the transmission error problem numerous techniques have been specified which are: forward error concealment, backward error concealment and interactive error concealment. In Forward error concealment technique the encoder plays the primary role. Backward error concealment refers to the concealment or estimation of lost information due to transmission errors in which the decoder fulfills the error concealment task. The decoder and encoder interactive techniques achieve the best reconstruction quality, but are more difficult to implement.

35 Error Resilience With the purpose of error concealment, scene signaling in SEI illustrates two kinds of information: (1) frames in which the shot change starts and ends; and (2) the type of the scene transition If a part of the current picture with which a scene information SEI message is associated is lost or corrupted, the decoder may apply a spatial error concealment algorithm to construct the lost or corrupted parts of the current picture if the scene has changed since the previous received picture. Otherwise the decoder may use a spatiotemporal error concealment algorithm.

. 36 Comparison between AVS Part2 and AVS Part 7 Profile Jizhun(AVS Part 2) Jiben(AVS Part 7) Available color formats 4:2:0, 4:2:2 4:2:0 Minimum block unit and 8 8 4 4 transform size Intra-Prediction 8 8 intra prediction 4 4 intra prediction Inter-Prediction Both P-Prediction and B- Prediction Only P-Prediction nonreference P Interpolation Two steps Four tap interpolation Two steps Four tap interpolation Maximum number of 2 2 reference frames Quantization Fixed quantization Fixed quantization Entropy coding 8 8 2D-VLC 4 4 2D-VLC Interlaced Support Frame coding or field coding Frame coding only Error resilience / Scene Signaling in SEI Table 1:Comparison between AVS Part 2 and AVS Part 7 [2]

37 Comparison between AVS Part7 and H.264 Baseline Profile Module AVS-M Jiben Profile H.264 Baseline Profile Intraluma prediction 9 4 4 modes Direct mode 9 4 4 modes 4 16 16 modes Intrachroma prediction 3 4 4 modes 4 4 4 modes Intraprediction Reference 9 samples 17 samples samples Interprediction 16 16 to 4 4 16 16 to 4 4 Subpixel interpolation 8-tap (1/2 horizontal), 4-tap 6-tap (1/2), linear (1/4) (1/2 vertical) Transform and quantization 4 4 ICT without scale in decoder 4 4 ICT with scale in decoder Entropy coding 2D-VLC Exp-Golomb code CAVLC Huffman/Exp- Golomb code Loop filter Each pixel is filtered once fewer pixels need filtering Each pixel is filtered once or twice. Table 2: Comparison between AVS Part 7 and H.264 Baseline Profile[2]

38 Conclusions and Future work AVS-M is an application driven coding standard with welloptimized and efficient techniques. It achieves performance similar those of H.264/AVC at a much lower cost. AVS Part 7 targets to low complexity, low picture resolution mobility applications. The AVS encoder and decoder are implemented using the AVS-M software. Tests are carried out on a set of QCIF and CIF sequences. The SNR values of the luma and chroma components are tabulated. The 2D-VLC can be further studied to improve the performance. The AVS-M access units are also a scope for study.

39 Results Original Sequence Plot of SNR vs Bits/frame for the Encoded Foreman Sequence 38 37 Decoded Sequence 36 35 34 33 14592 19192 8184 7616 7072 7320 7064 5944 5928 5816 5616 5328 5808 5624 4472 5376 5416 6128 5592 6576 SNR YUV in db Bits per frame

40 Results Original Sequence 38 Plot Of SNR vs Bits/frame for the Encoded News sequence 37 SNR YUV in db 36 35 34 33 17800 19584 3296 1016 4152 3392 4192 3912 1032 4056 4464 4216 3960 1408 3568 3664 3648 3688 1336 3936 Decoded Sequence Bits per frame

41 Results Plot Of SNR vs Bits/frame for the Encoded Mobile sequence 37 36 35 34 33 32 31 30 135720 163968 95096 86488 85392 86752 89344 87376 89328 89272 87112 90136 89032 90432 92432 91040 91152 90872 93520 89816 SNR YUV in db Bits/frame

42 Results Plot Of SNR vs Bits/frame for the Encoded Tempete sequence 37 36 35 34 33 32 31 90824 123832 69472 64640 62344 63632 63496 61048 61424 64088 62968 62560 63000 62672 68144 68696 70992 71576 68432 69976 SNR YUV in db Bits/frame

43 References [1] AVS working group official website, http://www.avs.org.cn [2] http://www-ee.uta.edu/dip/courses/ee5351/ispacsavs.pdf [3] L.Yu et al., Overview of AVS-Video: Tools, performance and complexity, SPIE VCIP, vol. 5960, pp. 596021-1~ 596021-12, Beijing, China, July 2005. [4] W.Gao et al., AVS the Chinese next-generation video coding standard, National Association of Broadcasters, Las Vegas, 2004. [5] L.Fan, Mobile Multimedia Broadcasting Standards, ISBN: 978-0-387-78263- 8, Springer US, 2009 [6] F.Yi et al., Low-Complexity Tools in AVS Part 7, J. Comput. Sci. Technol, vol.21, pp. 345-353, May. 2006 [7] L.YU, S.Chen and J.Wang, Overview of AVS-video coding standards, Signal Process: Image Commun, vol. 24, Issue 4, pp 247-262, April 2009 [8] W.Gao, AVS A project towards to an open and cost efficient Chinese national standard, ITU-T VICA workshop, ITU Headquarters, Geneva, 22-23 July 2005.

44 References [9] Z.Zhang et al., Improved Intra Prediction Mode-decision Method, Proc. of SPIE,Vol. 5960, pp. 59601W-1~ 59601W-9, Beijing, China, July 2005. [10] Z.Ma et al., Intra Coding of AVS Part 7 Video Coding Standard, J. Comput. Sci. Technol,vol.21, Feb.2006 [11] W.Gao and T.Huang AVS Standard -Status and Future Plan, Workshop on Multimedia New Technologies and Application, Shenzhen, China, Oct. 2007. [12] Y.Cheng et al., Analysis and application of error concealment tools in AVS-M decoder, Journal of Zhejiang University Science A, vol. 7, pp 54-58, Jan 2006. [13] M.Liu and Z.Wei A fast mode decision algorithm for intra prediction in AVS- M video coding Volume 1, ICWAPR apos;07, Issue, 2-4, pp.326 331, Nov. 2007. [14] Q.Wang et al., Context-Based 2D-VLC for Video Coding, IEEE Int l Conf. on Multimedia and Expo (ICME), vol.1, pp. 89-92, June. 2004. [15] http://vspc.ee.cuhk.edu.hk/~ele5431/avs.pdf

45 References [16] W.Gao, K.N. Ngan and L.Yu Special issue on AVS and its applications: Guest editorial, Signal Process: Image Commun, vol. 24, Issue 4, pp 245-344, April 2009. [17] S.W.Ma and W.Gao, Low Complexity Integer Transform and Adaptive Quantization Optimization, J. Comput. Sci. Technol, vol.21, pp.354-359, May 2006. [18] S.Hu, X.Zhang and Z.Yang, Efficient Implementation of Interpolation for AVS, Image and Signal Processing, 2008. Congress on Volume 3, Issue, 27-30 May 2008, pp133 138. [19] R. Schafer and T. Sikora, Digital video coding standards and their role in video communications, Proc. of the IEEE, vol. 83, pp. 907-924, June 1995. [20] A. K. Jain, Image data compression: A review, Proc. IEEE, vol. 69, pp. 349-384, March 1981. [21] T. Wiegand et al., Overview of the H.264/AVC Video Coding Standard, IEEE Trans. CSVT, Vol. 13, pp. 560-576, July 2003. [22] G.J. Sullivan, P. Topiwala and A. Luthra, The H.264/AVC advanced video coding standard: Overview and introduction to the fidelity range extensions, SPIE Conf. on applications of digital image processing XXVII, vol. 5558, pp. 53-74, Aug. 2004. [23] AVS China software can be downloaded from the following site ftp://159.226.42.57/public/avs_doc/avs_software