Fast Intra-Predcton Mode Decson n H.264/AVC Based on Macroblock Propertes Abstract Intra-predcton s a wdely used tecnque n ntra codng. H.264/AVC adopts rate-dstorton optmzaton (RDO) tecnque to obtan te best ntra predcton, wle maxmzng sual qualty and mnmzng te requred bt rate. Howeer, te RDO reduces te encodng speed a te exauste ealuaton of all canddate modes. In ts paper, we decrease te encodng tme by reducng te number of canddate modes. e proposed algortms use spatal propertes of macroblock (MB) (suc as drectonal edge nformaton), and some of statstcal propertes (suc as smlarty of erence pxels and adjacent blocks nformaton) to select a subset of all canddate modes. Subsequently, te RDO procedure uses te reduced subset of all canddate modes for extractng te fnal mode. Expermental results sow tat our algortm, compared to te RDO and some oter fast algortms, reduces te total encodng tme wt neglgble loss n PSNR and a slgtly ncreased btrate.. Introducton e newest deo codng standard s deeloped by te jont of deo teams of ISO/IEC MPEG and IU_ VCEG as te nternatonal standard 4496- (MPEG-4 part ) adanced deo codng (AVC) [, 2]. H.264/AVC as ganed more and more attenton; manly due to ts g codng effcency (te aerage btrate sang up to 5% as compared to H.26+ and MPEG-4 Smple Profle), mnor ncrease n decoder complexty compared to exstng standards, adaptaton to delay constrants (te low delay mode), error robustness, and network frendlness [, 2]. Fgure [] sow te performance comparsons usng MPEG-2, MPEG-4 (ASP), and H.264/AVC. o acee outstandng codng performance, H.264/AVC employs seeral powerful codng tecnques suc as 4x4 nteger transform, nter-predcton wt arable blocksze moton compensaton, moton ector of quarter-pel accuracy, n-loop deblockng flter, mproed entropy codng suc as context-adapte arable-lengt codng (CAVLC) and content-adapte bnary artmetc codng (CABAC), enanced ntra-predcton, multple erence pcture, and te fort. Fgure: Performance comparson of dfferent deo codng standards. Among many new features, te ntra-predcton tecnque s recognzed to be one of te man factors tat contrbute to te success of H.264/AVC. H.264/AVC employs te Lagrangan RDO metod to fnd out te best codng mode of ntrapredcton wt gest codng effcency. e ntra-predcton mode decson s ery complex and te number of computng RD cost alues for luma and croma of a macroblock s 592 [4]. o reduce te computatonal complexty, many algortms (suc as fast moton estmaton, fast nter-mode predcton, and fast ntra-predcton) ae been proposed. Fast mode selecton for ntra-predcton s consdered n ts paper; wc s a callengng subject n H.264/AVC, snce ntra-predcton s a new topc n H.264/AVC codng wt respect to oter standards suc as MPEG-/2/4 and H.26/H.26 and so far no preous work exsts for tat. Fast ntra-mode decson algortms usng edge detecton stogram and local edge detecton are proposed n [, 5, 6]. e performance of tose metods s about 2~% (or 55~65%) faster tan te RDO metod at te cost of 2% (or 5%) extra bts. ere exst fast algortms to select te optmal ntra-predcton mode usng smple drectonal masks n [7] wt sang tme of 7%, and statstcal-based metods n [8] wt sang tme of 45%. Anoter fast ntra-mode decson sceme s proposed n [9], were te encodng speed s approxmately % faster tan tat of te RDO metod. A new fast ntra-predcton algortm based on macroblock propertes (FIPAMP) s presented n []. s algortm can acee % to 4% of computaton reducton wle mantanng smlar PSNR and bt rate performance of H.264/AVC codes. In [], an effcent ntra-predcton (EIP) algortm based on early termnaton, selecte computaton of gly probable modes, and partal computaton of te cost functon s presented. Also, an mproed cost functon to mproe te codng performance s proposed n [2]. In [], a fast algortm based on te local edge nformaton obtaned by calculatng edge feature parameters, and subsamplng of matcng operatons s presented. at metod can reduce te encodng tme about 26% wt less tan.4% used extra bts and no more tan.7 db PSNR s sacrfced. In ts paper, we propose a metod for fast ntra-mode decson n wc te number of 4x4 and 6x6 ntra-mode for luma and 8x8 ntra-mode for croma as been reduced usng spatal features suc as drectonal edge nformaton, smlarty of erence pxels and spatal correlaton between current block and top-left blocks n I-frame. e proposed metod consumes less encodng tme by reducng te number of RDO computatons and by reducng te tme to obtan drectonal nformaton. We ae erfed te proposed algortm by mplementng t on JM7. erence software and comparng t wt te case of RDO searc. Smulaton results sow te proposed metod reduces te encodng tme up to 47% wt loss n PSNR and neglgble ncrease of requred btrate. e remanng parts of te paper are organzed as follows. We reew te ntra-predcton sceme of H.264/AVC and te mode selecton metod based on RDO tecnque n Sectons 2. Secton presents te new metods tat are proposed for fast ntra-mode decson m. Smulaton results are gen n Secton 4 and fnally Secton 5 concludes te paper.
2. Intra-Predcton n H.264/AVC Intra-predcton s based on te obseraton tat adjacent macroblocks tend to ae smlar propertes. Predcton may be formed for eac 4x4 luma block (I4MB), 6x6 luma MB (I6MB), and 8x8 croma block. For predcton of 4x4 lumnance blocks, te 9 drectonal modes consst of a DC predcton (Mode 2) and 8 drectonal modes; labeled,,, 4, 5, 6, 7, and 8 as sown n Fgure 2. In Fgure 2, te block (alues of pxels a to p ) s to be predcted usng A to Q. Note tat pxels A to Q from negborng blocks ae already been encoded and may be used for predcton. Fgure 2: Intra-predcton modes for 4x4 lumnance blocks. Labelng of predcton samples. Fgure sows a lumnance macroblock n a QCIF formatted frame and a 4x4 luma block tat s to be predcted. e samples aboe and to te left ae preously been encoded and reconstructed and are teore aalable n te encoder and decoder to form a predcton erence. e predcton block P s calculated based on te samples labeled A-Q. nter-predcton s done and RDO used for best predcton. e RDO procedure to encode one MB n an I-frame s gen n [4]. A part of proposed ntra-predcton algortm n JM7. s sown n fgure 4. Here, two sequences of QCIF and CIF formatted frames are encode usng JM7.. e results are sown for two I- and P-frame n Fgures 5 and 6, respectely. // Mode ; make DC predcton s = ; f (block_aalable_up && block_aalable_left) s = (A+B+C+D+I+J+K+L+ 4)/(8); else f (!block_aalable_up && block_aalable_left) s = (I + J + K + L + 2)/4; else f (block_aalable_up &&!block_aalable_left) // left edge s = (A + B + C + D + 2)/4; else //f (!block_aalable_up &&!block_aalable_left) // top left corner, notng to predct from s = 28; for (j=; j < 4; j++) for (=; < 4; ++) // store DC predcton mg->mprr[dc_pred][][j] = s; // Mode DIAG_DOWN_RIGH_PRED f (block_aalable_up && block_aalable_left && block_aalable_up_left) m = (L + 2*K + J + 2) / 4; =n = (K + 2*J + I + 2) / 4; e =j =o = (J + 2*I + Q + 2) / 4; a=f=k=p= (I + 2*Q + A + 2) / 4; b=g=l = (Q + 2*A + B + 2) / 4; c== (A + 2*B + C + 2) / 4; d = (B + 2*C + D + 2) / 4; Fgure 4: A part of proposed ntra-predcton algortm n JM7. [4]. Fgure : 7st frame of orgnal walkng person sequence. Orgnal macroblock. (c) 4x4 luma block to be predcted [5, 6]. For regons wt less spatal detals (.e., flat regons), H.264/AVC supports 6x6 ntra-codng; n wc one of four predcton modes, DC (mode 2), ertcal (mode ), orzontal (mode ) and planar (mode ) s cosen for te predcton of te entre lumnance component of te macroblock[7], as: Also,H.264/AVC supports four croma predcton modes for 8x8 cromnance blocks, smlar to tat of te I6MB predcton, except tat te order of mode numbers s dfferent: DC (Mode ), orzontal (Mode ), ertcal (Mode 2), and plane (Mode ). e same predcton mode s always appled to bot croma blocks. e croma predcton s ndependent from luma predcton. Fnally, for I-frames, wle all MBs are predcted as Intra, H.264/AVC encoder encodes te best mode usng all mode combnatons of luma and croma and cooses te one tat ges te best RDO performance. For P-frames, ntra- and Fgure 5: I-frame of foreman sequence (QCIF). I4MB and I6MB predcton mode decson. I4MB and I6MB dsons.
d) e optmal mode (found by full searc) and oter good (second or trd best) modes are most lkely n smlar drectons. e) e drecton features of 4x4 blocks can be presered rougly after down-samplng. f) ere are a total of erence pxels for ntra predcton of a 4x4 luma block, wc locate at te up and te left of te 4x4 luma block. Expermental results sow tat te erence pxels of a 4x4 luma block are smlar wt eac oter wt a g probablty [2]. Based on tese obseratons, we propose a new fast mode selecton for I4MB and I6MB.. Fast Mode Selecton for 4x4 Luma Block Usng Subsamplng and Edge Informaton o extract te local edge nformaton, te algortm ntroduced n [] ddes a 4x4 block nto four 2x2 blocks. Usng A, B, C, and D to denote te sum of ntensty of all pxels n te correspondng 2x2 blocks, gen as: A = C = = = j= = 2 Pj Pj B = D = j = 2 = j= 2 = 2 Pj Pj () Fgure 6: P-frame of football sequence (CIF). I4MB and I6MB (yellow) and nter-predcton (blue) mode decson. I4MB, I6MB, and nter-mode dsons.. Proposed Metods for Fast Intra-Predcton Mode Decson s secton presents a new fast algortm tat s based on statstcal propertes of MBs. s s motated by some obseratons n our experments. e proposed metod s based on seeral facts tat we obsered from te statstcs of dfferent sequences tat: a) For ntra-predcton of lumnance samples te probablty of 4x4 block sze s sgnfcantly ger tan 6x6 block sze at usual quantzaton parameters (2~5), as sown n fgure 7. eore fast detecton of 4x4 ntra-predcton mode can sgnfcantly mproe te encodng speed at low QP, wle 6x6 ntra-predcton at large QPs. 6 5 6x6 MB 4x4 MB In order to obtan te local edge drecton wtn a 4x4 block, ts work ntroduces two edge feature parameters: ertcal edge parameter F and orzontal edge parameter F as: ( A + B) ( C + D) FV = (2) S ( A + C) ( B + D) F = S Were S s a scalng factor []. Accordng to tese two parameters, te edge drecton nformaton wtn te current 4x4 block can be obtaned. able sows te results. On te oter and, te fast metod only cooses te predcton mode along te edge as te canddates of te best predcton modes (CBPM). Accordng to F and F, te determned dfferent CBPMs for te current block correspondng to 7 cases (see able ) are obtaned and summarzed n able 2. able : Edge drecton nformaton. 4 Number of MB 2 5 2 25 5 4 45 5 Quantzaton Parameter Fgure 7: Number of 4x4 and 6x6 ntra-coded macro blocks at dfferent quantzaton parameters. b) e predcton modes of eac block are correlated wt tose of negborng 4x4 lumnance blocks. e statstcs generated usng JM 7. encoder [4] sows tat for a wde arety of nputs large negborng blocks ae te same I4MB mode c) Normally, pxels along te drecton of local edge ae smlar alues. In [], a fast ntra-mode decson metod s proposed, wc s based on edge detecton usng te Sobel operaton and edge drecton stogram (EDH). able 2: CBPMs accordng to F and F. Case 2 4 5 6 7 CBPMs 2,2,2,2 4,2,5,7,4,2,6,8,4,2
Snce te number of modes for case 6 and 7 are large and DC mode exsts n all cases, we modfy ts metod usng smlarty of erence pxels and correlaton of adjacent blocks. Also, we modfed ts algortm for I6MB luma and 8x8 croma n Subsecton.2 and. to mproe te proposed algortm. Based on tese facts, te steps of te proposed metod are as follows:. Calculate te most probable ntra-predcton mode for 6 and 7 cases (n table 2), usng Pan s metod []. s s te maxmum of edge detecton stogram and s called MEDH. 2. For 4x4 luma block, MAD (mean of absolute dfference) of ts erence pxels s computed, f t s smaller tan a tresold, DC mode s rejected from te cases of 2,, 4 and 5 and only one mode s selected for tese cases. Also for 6 and 7 te MEDH s selected. Go to step 7. s result s yelded from ts fact tat f te smlarty of erence pxels of a block s g, te dfference between dfferent predcton modes wll be ery small. For ts case, t s not necessary to ceck all 9 predcton modes, but only one predcton mode s enoug [2].. If MADH (mean of absolute dfference of orzontal erences) s less tan a tresold and MEDH s a member of set mode, mode, mode 7, ten MEDH s selected. Go to step 7. 4. Also, f MADV (mean of absolute dfference of ertcal erences) s less tan a tresold and MEDH s a member set of mode, mode 8, ten MEDH s selected. Go to step 7. It s obous tat f te smlarty of orzontal erence pxels of a block s g, te dfference between predcton results obtaned wt predcton modes, and 7 wll be ery small. Also, f te smlarty of ertcal erence pxels of a block s g, te smlarty between modes and 8 s g. 5. If te modes for one of te top or te left blocks (M_A, M_B) are MEDH, ten Mode MEDH s cosen as te best canddate mode for te current block. Go to step 7. 6. At last f all of aboe condtons are unsatsfed, te Pan s metod s used for 6 and 7 cases []. For 2,, 4 and 5 te algortm [] s suffcent. 7. ermnate. Expermental results sow, tat n many cases only one mode wll be ealuated..2 I6MB based on Horzontal and Vertcal Dfferences (HVD) Let and denote te ertcal and orzontal sum of dfferences between boundary pxels of current block and ts adjacent block, respectely [7]: = 5 = 5 u cu, = l cl () = were: u : boundary pxels of upper MB cu : upper boundary pxels of current MB l : boundary pxels of left MB cl : left boundary pxels of current MB s metod obtans canddate modes by usng two dfferences alues, under te condtons as follows: - - > canddate modes are DC mode and orzontal mode. 2- - > canddate modes are DC and ertcal mode. - - < canddate modes are DC mode and plane mode. 4- Fnally, determne te best mode among canddate modes by coosng te mode tat results n te mnmum SAD. For reducng te encodng tme we can use te fast Hadamard procedure for SAD computaton.. Fast Mode Selecton for 8x8 Croma Block For 8x8 croma blocks, we apply a smlar metod to te metod used for 6x6 luma macroblock..4 Early ermnaton of RDO Calculaton Smlar to Pan s metod, for ncreasng te speed of te algortm we use te early termnaton of RDO calculatons for all proposed algortms as n []. Durng te ntra-codng of any predcton mode, te calculaton can be termnated f t can foresee tat te current mode wll not be te best predcton mode. By early termnaton of te RDO calculatons wc s deemed to be suboptmal, a great tmesang can be aceed. In te RDO, te codng cost conssts of two parts: rate and dstorton. After calculatng te cost of rate, tere mgt be cases tat te cost of rate s ger tan te codng cost of te best mode n te preous modes. s mples tat te current mode wll not be te best mode snce ts codng cost wll not be te smallest. eore, n suc cases we termnate te RDO calculaton and ence te calculaton of dstorton wll be elmnated. An MB s encoded by eter I4MB or I6MB predctons. In te RDO, te selecton between tese two codng modes s determned by te codng costs aceed by usng eac of tese codng modes. After I6MB predcton codng, te I4MB predcton codng wll apply to te sxteen 4x4 blocks n te MB and te cost of tese blocks wll be accumulated. Howeer, f te accumulated cost before encodng te entre sxteen 4x4 blocks s already ger tan tat of I6MB predcton codng, te codng of te remanng of 4x4 blocks n te MB wll be termnated prematurely. 4. Expermental Results Our proposed algortm was mplemented nto JM7.[4], proded by JV accordng to te test condtons specfed n VCEG-N8 [2] document as lsted n able. Smulatons were carred out on te recommended sequences wt arous quantzaton parameters for IPPP type and I-frame only type. For IPPP experments, te total number of frames s for eac sequence, and te perod of I-frame s. e used test platform s Pentum IV-2.8 GHz wt 256 Mbytes RAM. able : Experment condtons. GOP IIIII or IPPP Codec JM 7. MV searc range ± 6 QP, 6, 24,28,6,4 Number of Reference Common codng opton Hadamard transform, CABAC Sze CIF and QCIF Number of Frames
Comparsons wt te case of exauste searc (RDO) were performed wt respect to te cange of aerage PSNR ( PSNR), te cange of aerage data bts ( Bt), and te cange of aerage encodng tme ( me), respectely. e PSNR s dered from aerage PSNRs of luma component (Y) and croma component (U, V) based on []. Also, n order to ealuate te tme sang of te proposed fast ntra-mode decson algortm, te followng calculaton s defned to fnd te tme dfferences. Let denote te codng tme used by JM7. encoder and prop be te tme taken by te faster ntra-predcton algortm, and tme be defned as: me % = prop (4) Also, btrate ncrease s defned as: btrate prop btrate Btrate % = btrate (5) A group of experments were carred out on dfferent sequences and te results are sown below. e experments were ordered n 4 states as lsted n able 4. e encodng bt rates, te PSNR alues, and te tme sang factor (as compared wt te H.264 RDO metod) for 4 test sequences wt dfferent quantzaton parameters are sown n ables 5~. Also, ables 5~7 and 8~ sow te expermental results for IPPPP and IIII sequences, respectely. ese tables compare te rate, dstorton, and complexty of proposed algortms wt RDO procedure. Generally speakng, as can be seen from ts tables, we ae saed 42~57% of te total encodng tme at te expense of only.2% rate ncrease n aerage and -.8dB dstorton loss n aerage for tese test sequences. Fgures 8 ow te examples of RD and te complexty cures of sequences Akyo for IPPP sequences. From ts fgure, one can see tat te proposed fast ntra-mode decson sceme ges almost dentcal RD performance wle prodng a g speed-up factor (rato of encodng tme usng te RDO tecnque and te proposed sceme). In ts fgure te RDO, Pan s metod, Algortm [], and te fast proposed metods are compared (see able 4). We see tat for proposed algortms te rate-dstorton performance loss ncreases slgtly wt a g complexty reducton. able 4: Dfferent metods of experment. Category I4MB I6MB, Early Croma ermnaton RDO RDO RDO NO M Pan s Metod RDO Yes M Algortm [] RDO Yes M4 New proposed RDO Yes metod In ts paper usng statstcal propertes of natural deo sequences for fast ntra-mode decson n te H.264/AVC encoders, we decreased te encodng tme by reducng te number of canddate modes. In order to acee a better performance of computatonal complexty some new deas are combned, and a new algortm s presented. e smulaton results sow tat te proposed algortm reduces te number of RDO calculatons wt respect to orgnal algortms wt neglgble loss n PSNR and neglgble btrate ncrease. e proposed algortm can be used for callengng work of ntrapredcton mode decson n te H.264/AVC deo encoders wt low computatonal cost. able 5: Smulaton results for IPPP type sequences. Dstorton comparson able 6: Smulaton results for IPPP type sequences. Rate comparson 5. Concluson
able 7: Smulaton results for IPPP type sequences. Complexty comparson able. Smulaton results for IIII type sequences. Complexty comparson able 8: Smulaton results for IIII type sequences. Dstorton comparson able 9. Smulaton results for IIII type sequences. Rate comparson Fgure 8: Akyo sequence, (IPPP seq.). R-D performance, computatonal complexty.
References [] F. Pan, X. Ln, S. Raardja, K. P. Lm, Z. G. L, D. Wu, and S. Wu, Fast Mode Decson Algortm for Intra-predcton n H.264/AVC Vdeo Codng, IEEE rans. On crcuts and systems for deo ec., Vol. 5, NO. 7, pp. 8-822, July 25. [2]. Wegand, G. J. Sullan, G. Bjontegaard, and A. Lutra, Oerew of te H.264/AVC Vdeo Codng Standard, IEEE rans. On crcuts and systems for deo tecnology, Vol., no., no. 7, pp. 56-576, 2. [] Eno ttp://www.eno.com/products/264.tml [4] Cangsung Km, Qng L, C. C. Jay Kuo, Fast Intra-Predcton Model Selecton for H.264 Codec, SPIE Internatonal Symposum ICOM 2, Orlando, Florda, Sept. 7-, 2. [5] F. Pan, X. Ln, et al., Fast Mode Decson for Intra- Predcton, ISO/IEC JC/SC29/WG and IU- SG6 Q.6, JV 7t Meetng, Pattaya II, aland, Marc 2. [6] F. Pan, X. Ln, S. Raardja, K. P. Lm, and Z. G. L, A Drectonal Feld Based Fast Intra-Mode Decson Algortm for H.264 Vdeo Codng, IEEE Inter. Conf. on Multmeda and Expo, ol. 2, pp. 47-5, June 24. [7] J. Km and J. Jeong, Fast Intra-Mode Decson n H.264 Vdeo Codng Usng Smple Drectonal Masks, Proc. of SPIE Vol. 596, pp 7-79, 25. [8] R. Garg, M. Jndal, M. cauan, Statstcs Based Fast Intra- Mode Detecton, Proc. of SPIE Vol. 596, pp 285-29, 25. [9] B. Jeon and J.lee, Fast Mode Decson for H.264, ISO/IEC JC/SC29/WG and IU- SG6 Q.6, JV t Meetng, Wakoloa, Hawa, December 2. [] C. Yang, L. PO, W. Lam, A Fast H.264 Intra Predcton Algortm Usng Macroblock Propertes, ICIP, pp. 46-464, 24. [] B. Meng, O. C. AU, C. Wong, H. Lam, Effcent Intra- Predcton Algortm n H.264, IEEE, pp. 87-84, 2. [2] C. seng, H. Wang, J. Yang, Improed and Fast Algortms for Intra 4x4 Mode Decson n H.264/AVC, IEEE, pp. 228-2, 24. [] C. Hsu, M. Ho, J. Hong, An Effcent Algortm for Intrapredcton n H.264, IEEE, pp. 5-6, 26. [4] Jont Vdeo eam (JV), erence software JM7., ttp://bs//.de/~suerng/tml/download/jm7..zp. [5] M. Jafar, S. Kasae, An Effcent Intra Predcton Mode Decson Algortm for H.26 to H.264 ranscodng, IEEE nternatonal conference on computer systems and applcatons, page 82-89, marc 26. [6] M. Jafar, s. Kasae, Prortsaton of data parttoned MPEG-4 deo oer GPRS/EGPRS moble networks, Asan nternet engneerng conference (AINEC), aland, pp. 68-82, December 25. [7] I. Rcardson, H.264/MPEG-4 part wte paper, aalable: ttp://www.codex.fsnet.co.uk/resources.tml. [8] C. Km, H.-H. S, and C.-C. J. Kuo, Feature-based ntrapredcton mode decson for.264, n Proc. IEEE Internatonal Conference on Image Processng, 24. [9] Cen Cen and Png-Hao Wu, Hornor Cen, ransformdoman ntra predcton for H.264, IEEE, pp. 497-5, 25. [2] G. Sullan, Recommended smulaton common condtons for H.26L codng effcency experments on low resoluton progresse scan source materal, presented at te 4t VCEG- N8 Meetng, Santa Barbara, CA, Sep. 2. [2] Jang Gang-y, L S-png, Yu Me, L Fu-cu, An effcent fast mode selecton for ntra predcton, IEEE Int. Worksop VLSI Desgn & deo ec., Cna, pp. 57-6, May 28-, 25.