TRANSFORM DOMAIN SLICE BASED DISTRIBUTED VIDEO CODING

Journal of Engineering Science and Technology Vol. 6, No. 5 (2011) 542-550 School of Engineering, Taylor s Universiy TRANSFORM DOMAIN SLICE BASED DISTRIBUTED VIDEO CODING A. ELAMIN*, VARUN JEOTI, SAMIR BELHOUARI Elecrical Eng., Uinevrsii Teknologi PETRONAS, Perak, Malaysia *Corresponding Auhor: eaea33@iu.dk Absrac Disribued video coding depends heavily on he virual channel model. Due o he limiaions of he side informaion esimaion one saionary model does no properly describe he virual channel. In his work he correlaion noise is modelled per slice o obain locaion-specific correlaion noise model. The resuling delay from he lenghy Slepian-Wolf (SW) codec inpu is also reduced by reducing he lengh of SW codec inpu. The proposed soluion does no impose any exra complexiy, i uilizes he exising resources. The resuls presened here suppor he proposed algorihm. Keywords: Video Coding, Slepian-Wolf, Transfer Domain, Noise Model. 1. Inroducion Today s digial video coding paradigm represened by he ITU-T and MPEG sandards mainly relies on a hybrid of block-based ransform and iner-frame predicive coding approaches. In his coding framework, he encoder archiecure has he ask o exploi boh he emporal and spaial redundancies presen in he video sequence, which is a complex process and i requires a noiceable amoun of resources (Power and memory), while he decoder remains a pure execuer of he encoder insrucions. As a resul, all sandard video encoders have a much higher compuaional complexiy han he decoder (ypically five o en imes more complex) [1, 2]. Recenly new emerging applicaions such as wireless lowpower surveillance and mulimedia sensor neworks, wireless PC cameras and mobile camera phones have very differen requiremens han hose of radiional video delivery sysems. For some applicaions noably when here is a high number of encoders and only one decoder, e.g., surveillance, low cos encoder 542

Transform Domain Slice Based Disribued Video Coding 543 devices are necessary. I is essenial o have a low-power and low-complexiy encoder device, possibly a he expense of a higher complexiy decoder. The low-complexiy encoding could be achieved by moving some of he encoder asks o he decoder par, specially he mos complex moion esimaion process. This useful hin is he consequence of informaion-heoreic principles esablished in he 1970s by Slepian and Wolf for disribued lossless coding [3], and by Wyner and Ziv for lossy coding [4] wih some side informaion made available a decoder. Schemes ha are based on hese heorems are generally referred o as disribued source coding algorihms. The aracive idea of disribued source coding is ha, in he case of join decoding, wo correlaed sources X and Y can be compressed separaely wihou he knowledge of he oher source. I can sill aain he same compression as if he oher source was known. For he specific case where Y is available a he decoder and X and Y are joinly Gaussian, Wyner [4] proved ha by using channel coding a he encoder he compression can sill be achieved wihou Y being known a he encoder, since he availabiliy of a correlaed source is only necessary a he decoder. Designing pracical video codec uilizing hese principles in disribued video coding is sill in is infancy, and researchers all over he world are eager o explore i and propose video coding sysems. One of he firs pracical WZ video coding soluions has been developed a Sanford Universiy [1, 2, 5] works a he frame level. In his work i will be referred o by frame based soluion. This soluion has become he mos popular WZ video codec design in he lieraure. The basic idea of his WZ video coding archiecure is ha he decoder, based on some previously and convenionally ransmied frames, he so-called key frames, creaes he so-called SI which works as esimaes for he oher frames o code, he so-called WZ frames. The WZ frames are hen encoded using a channel coding approach, e.g., wih urbo codes or Low-Densiy Pariy-Check (LDPC) codes [5, 6], o correc he esimaion errors in he corresponding decoder esimaed side informaion frames. In his case, he encoding is performed assuming here is (high) correlaion beween he original WZ frames o encode and heir associaed SI frames a he decoder; higher correlaion leads o more efficien encoding process. In his paper, secion wo reviews he sae-of-he-ar ransform domain WZ codec and presens he problems and issues relaed o he sae-of-he-ar he secion also presens some relaed works ha address he idenified issues. Secion hree inroduces he proposed slice based ransform domain DVC. The slice composing mehod is presened in secion four. The experimenal work and is resuls are presened in secion five. The curren work and he fuure work are concluded in secion six. 2. Lieraure Review Figure 1 shows he sae-of-ar ransform domain WZ video coding as presened in [1, 2, 7, 9]. The overall coding archiecure works as follows: he video sequence is divided ino WZ frames and key frames; he key frames are H263+ inra coded. Over each WZ frames a 4-by-4 is applied. The coefficiens of he enire frame are grouped ogeher in bands. Each band is uniformly quanized and biplanes are exraced and sen o he urbo encoder. The

544 A. Elamin e al. urbo encoding process for a given band sars wih he mos significan biplane. Only a fracion of he pariy informaion generaed by he urbo encoder for each biplane is sen o he decoder. The decoder he frame inerpolaion module is used o generae he side informaion Y ; an esimae of he X frame, based on he previously decoded frames X +1 and X -1. For a GOP lengh 2, hese wo frames are he key frames. A 4-by-4 is hen carried ou over Y in order o obain Y, an esimae of X. The residual saisics beween corresponding X and Y is assumed o be modeled by a Laplacian disribuion. The Laplacian parameers are esimaed online for each coefficien based on he residual beween he frames X + 1 and X 1 afer moion compensaion. Once he and he residual saisics for a given band are Y know he decoded quanized symbol sream associaed o ha band can be obained hrough an ieraive urbo decoding procedure. Afer urbo decoding he mos significan biplane of he he urbo decoder proceeds in analogous way o he remaining biplanes associaed o ha band. Once all he biplane arrays of a given band are urbo decoded he urbo decoder sars decoding he nex band. This procedure is repeaed unil all he bands for which WZ bis are ransmied are urbo decoded. Afer urbo decoding he biplanes associaed o a given band, hese biplanes are grouped ogeher o form he decoded quanized symbol sream associaed o ha band. Once all he decoded quanized symbols are obained i is possible o reconsruc he marix of coefficiens Xˆ. For some bands no WZ bis are ransmied; a he decoder hose bands are replaced by he corresponding bands of SI Y. The remaining bands are obained using he reconsrucion funcion which bounds he error beween coefficiens of X and Xˆ (also known as reconsrucion disorion) o he quanizer coarseness. Fig. 1 Sae-of- ar Transform Domain WZ Codec Archiecure. This secion inends o invesigae some issues in he sae-of-he-ar such as complexiy of he decoding process and he accuracy of he correlaion noise model and review he lieraure relaed o hese issues addressed in his paper. The size of inpu block is assumed arbirarily large because i represens all bis from he whole frame or one bi-plane a once [7-9]. Too large inpu block will produce

Transform Domain Slice Based Disribued Video Coding 545 significan compuaion laency during he encoding and decoding process. In oher words, he sysem will no be able o provide a imely WZ decoded oupu due o he vas Slepian-Wolf urbo coding and decoding delay. Frame based DVC sysems make use of a rae-compaible-puncured codes (RCPT) codec as he module of he SW codec [10, 11] and he puncure window size is usually 8, resuling in 16 puncure code raes, which is referred o as rae conrol levels. The encoder blindly sends he pariy bis according o he puncuring paern deermined by nex lower coding rae. This causes he wase of ransmission ha some unnecessary pariy bis are sen o help decoding some bis which are already correcly decoded. The decoding will sill reques more pariy bis unil enough pariy bis are received. I can be seen ha by dividing he WZ frame ino several small pars for ransmission, he ransmission of pariy bis is no aiming o decode he whole frame bu a small par of he frame herefore reduces he possibiliy o ransmi he unnecessary pariy bis. In radiional video coding, he Laplacian disribuion is ypically used o model he disribuion of he moion-compensaed residual coefficiens [12]. More accurae models can be found in lieraure, such as he generalized Gaussian disribuion; however, he Laplacian disribuion consiues a good rade-off beween model accuracy and complexiy and, herefore, i is ofen chosen [7-9, 13]. For he same reason, mos of he implemenaions of he frame based DVC assume saionary noise (Laplacian) and consan qualiy of he side informaion esimaed a he decoder along he frame. In disribued video compression, he side informaion is acually esimaed a he decoder side; he decoder has o make a predicion of he curren video frame wihou acually knowing his frame. As a consequence here will be wihin he moion prediced frame co-exis regions where he moion esimaion is successful (high correlaion) and a region where he inerpolaion has (more or less) fails due o he occlusion. Occlusions creae noise in moion esimae frame predicion wih wo imporan properies. Firsly, his noise is very locaion-specific; and always locaed a he edge of a moving objec or he frame edge in he case of camera moion; Secondly, occlusion noise is hard o characerize [14]. Disribued source coding relies heavily on efficien error correcing codes; he performance of hese codes depends grealy on he choice of he noise model ha characerizes he virual dependency channel [15]. I was concluded ha he behavior of he virual dependency channel is subsanially more complicaed han a simple BSC or AWGN channel model ofen assumed in communicaions sysems ha apply channel codes [16]. Several works has been proposed for frame-based in order o cope wih above wo drawbacks. In [18], in order o exploi he spaial variabiliy of he correlaion noise, he models parameer is calculaed a he block level; he block size considered is equal o he one used in he frame inerpolaion process (8-by- 8) in order o more easily mach he frame inerpolaion errors; he resuling RD performance is ouperformed he frame level based noise channel modeling. Nonsaionary model o characerize he correlaion noise is proposed in [14], where wo models are used one o model he non-occluded regions and he oher o model he occluded regions, he resuls shows ha he performance of he SISO decoder, and herefore of he overall sysem, can be improved grealy by classifying he decodergeneraed side-informaion ino wo (or more) reliabiliy classes. I also shows ha he channel model should be an accurae represenaion of he real behavior of he channel; oherwise he decoding performance will heavily degrade. The lenghy

546 A. Elamin e al. block problem has also been addressed in [19, 20], by dividing he WZ frame X ino M sub-images X m, m = {1, 2,, M}each sub-image is independenly encoded using a Turbo-code based Wyner-Ziv encoder. Therefore, he block size of he Turbo encoder decreases o 1/M of he frame size. In he proposed sysem [19], BAWZCbased DVC sysem, he puncuring rae is independenly adjused for each block and he BAWZC decoder has o perform o inform he WZC encoder which blocks needs more pariy bis. The sysem proposed in [20] resrics he lengh of SW inpu o reduce he decoding complexiy and he resuling delay. Boh work assume he saionary noise and use he same correlaion noise model o characerize he noise over all sub-images, which makes no poin of he sub-image oher han jus reducing he lenghy block. They alleviae he problem of spreading he esimaion errors all over he bisream, by keeping he localized errors close o each oher in he generaed symbol sream. In he resuls of boh works, i is observed ha sysem performance is furher enhanced. 3. Proposed DVC Sysem For a video sequence, he odd frames are he Key frames, and he even frames are he Wyner-Ziv frames. The Key frames can be inraframe encoded by using any convenional video codec and inraframe decoded a he decoder wih he same codec. In he proposed DVC scheme, he Wyner-Ziv frames are inraframe encoded by using a slice Wyner-Ziv codec. However, hey are inerframe decoded by he proposed slice based decoder joinly wih he side informaion which is generaed from he corresponding key frames. The WZ frame is divided ino slices based on a binary map generaed by he decoder. The decoder composes he slice and generaes binary map o help he encoder compose he corresponding slices from he original WZ frame. The decoder assesses he degree of success in he generaion of side informaion frame blocks, he background blocks are grouped ogeher o compose he firs slice. The Occluded regions blocks grouped ogeher o compose anoher slice; he remaining blocks are grouped ogeher as anoher slice. The resuling slices will namely be he background slice, he simple moion slice and he occluded regions slice. The decoder generaes he corresponding side informaion for each slice and ransforms hem ino domain. The coefficiens are grouped ino sub bands, he correlaion noise model for each sub band is esimaed; he resuling model is more accurae han using he single model o describe he noise over sub band formed o he enire frame and i is a locaion-specific model. p ( X Y ) slice slice α = 2 slice e α slice X slice Yslice The feedback channel plays an imporan role in urbo-code based DVC sysems [1, 2, 7, 19]. In he proposed sysem, he Slice based DVC decoder has o perform no only rae conrol bu also he decoder uses he feedback channel o send he binary map o guide he encoder o compose he corresponding original slices. Afer WZ frame is divided ino slices a he Slice based DVC encoder, a 4-by-4 is applied o each slice, each sub band is hen quanized using quanizaion scheme as proposed in [9], each quanized sub band represened as a se of biplanes according o he number of he quanizaion levels of he sub band corresponding quanizer. The resuling binary sequence is hen fed ino a urbo encoder as a symbol sream. (1)

Transform Domain Slice Based Disribued Video Coding 547 Afer urbo encoding, all sysemaic bis are discarded and all pariy bis are sored in a buffer. The pariy bis are progressively requesed by he SWZC decoder o perform he rae conrol process [1, 2, 7, 9]. 4. Slice Composing The decoder performs he blocks classificaion o slice he side informaion and help he encoder creae he corresponding slices, o avoid add more complexiy o he decoder he process of slicing he side informaion and generaing he binary map is embedded wihin he process of frame inerpolaion. The moion esimaion/compensaion process is performed a he decoder in which he moion vecors forward/backward are esimaed, he background slice is simply composed by zero forward/backward moion vecors blocks. The same informaion (forward/backward moion vecors) is used o deec he occluded regions blocks. The usual assumpion in esimaion of occlusions from wo frames [21, 22], is excessive inensiy maching (moion-compensaed predicion) error observed; reference-frame pixels ha disappear canno be accuraely mached in he arge frame and hus induce significan errors. Le F 1 (x) denoe inensiy of he firs frame of a sequence a spaial posiion, and F 2 (x) similar inensiy in he second frame. If d f (x) denoes a forward moion (dispariy) field anchored on he sampling grid of key frame #1 (reference) and poining o he arge key frame #2, while d b (x) denoes a backward moion field, hen he corresponding moioncompensaed predicion errors a x are: 2 ( f ) ( x d x ) ( x) F x d ( x) f x = F1 + (2) 1 ( x) F ( ) b x = F + (3) 2 f The usual occlusion deecion mehods hen declare a pixel in he reference frame as being occluded in he arge frame if f >ε or frame #1 and b >ε for frame #2. Noe ha alhough newly exposed areas canno be deeced by his mechanism (pixels are no visible), effecively he occluded areas in frame #2 are in fac he newly-exposed areas for frame #1. 5. Experimens and Resuls To es he performance of he proposed DVC scheme, he firs 101 frames of he QCIF (144 x 176) Foreman and Hall Monior sequences a 15 frames per second were coded.. The occlusion deecion based on he phoomery based mehod (equaion 2, 3) is performed a he decoder during he side informaion creaion, he hreshold value for he occluded regions is obained empirically and hardcoded as 253. The frame inerpolaion process is performed as in [5]. From Table 1, he lengh of he biplane is reduced; as resul he corruped par of he original biplane is isolaed in differen biplanes (biplanes of slice 2 and slice 3). Table 2 shows he correlaion noise parameers for same sub bands in differen slices, he differen parameers correspond o he difference in he qualiy of he side informaion along he esimae frame. Figures 2 and 3 show he resuling rae-disorion performance of he proposed sysem for he Wyner-Ziv frames of he Foreman and Hall Monior sequences, respecively.

548 A. Elamin e al. Table 1. The Biplane Lengh per Slices. Video sequence Slice 1 Slice 2 Slice 3 lengh of biplane lengh of biplane lengh of biplane Hall Monior 1264 280 40 Foreman 824 412 348 Table 2. The Correlaion Noise Model Parameer per Sub-band/Slice. Foreman Sequence Hall Monior Sequence Slice Slice DC AC1 AC2 Number Number DC AC1 AC2 1 0.2267 0.5661 0.6705 1 0.2348 0.4318 0.4364 2 0.2301 0.4750 0.4629 2 0.2385 0.2627 0.3630 3 0.2090 0.3617 0.3608 3 0.2991 0.4329 0.3105 Fig. 2. PSNR Comparison of he Proposed Sysem for he Hall Monior Sequence. Fig. 3. PSNR Comparison of he Proposed Sysem for he Foreman Sequence. For comparison, Figs. 2 and 3 also show he corresponding rae disorion performance of he ransform-domain DVC sysems of [5] and of H.263 inerframe coding (P frames). From Figs. 2 and 3, i can be seen ha he proposed sysem (Slice based) RD performance is closer o H.263 inerframe coding han exising DVC sysems [5]. In addiion, he proposed DVC sysem resuls in a 25% o 35% reducion in he average birae for he Foreman sequence, and in an

Transform Domain Slice Based Disribued Video Coding 549 average bi-rae reducion of 21% o 30% for he Hall Monior sequence, as compared o [5]. In he rae disorion performance he rae of he feedback is no considered as exra rae since i flows from he decoder o encoder. The same reconsrucion mehod as in [5] is applied on boh as he rae disorion figures show same qualiy PSNR for boh soluions, wih differen raes. 6. Conclusions The design of he slice srucure opimizes usage of he exising resources. I is found ha he uilizaion of slice srucure in WZ video coding brings wo advanages. On one hand, he inpu block size is reduced and non-saionary model properly characerizes he virual channel. On he oher hand, wih slice srucure, he encoder avoids sending unnecessary pariy bis as a resul he RD performance is improved as a whole. The gap beween convenional inerframe codec and he disribued video coding is reduced and furher narrowing o his gap is possible by improving he side informaion esimaion. The process of deecing he occluded regions does no increase he complexiy of he decoder since i is performed wihin he frame inerpolaion process; communicaing he slicing binary map uilizes he exising feedback channel; herefore he proposed slice based DVC does no impose exra complexiy o he overall sysem. Fuure work is o devise advanced mehod o exrac he background blocks and replace hem wih he key frames blocks. References 1. Aaron, A.; Zhang, R.; and Girod, B. (2002). Wyner-Ziv coding of moion video. The Thiry-Sixh Asilomar Conference on Signals, Sysems and Compuers, 1, 240-244. 2. Aaron, A.; and Girod, B. (2004).Wyner-Ziv video coding wih low-encoder complexiy "Invied Paper". In Proc. Picure Coding Symposium, PCS-2004, San Francisco, CA. 3. Wyner, A.; and Ziv, J. (1976). The rae-disorion funcion for source coding wih side informaion a he decoder. IEEE Transacions on Informaion and Theory, 22(1), 1-10. 4. Aaron, A.; and Girod, B. (2002). Compression wih side informaion using urbo codes. IEEE Conference on Daa Compression, 252-261. 5. Varodayan, D.; Aaron, A.; and Girod, B. (2005). Rae-adapive disribued source coding using low-densiy pariy-check codes. Conference Record of he Thiry-ninh Asilomar Conference on Signals, Sysems and Compuers, 1203-1207. 6. Rebollo-Monedero, D.; Aaron, A.; Girod, and B. (2003). Transforms for high-rae disribued source coding. In Proceedings of 37 h Asilomar Conference on Signals, Sysems and Compuers, 1, 850-854. 7. Ascenso, J.; Bries, C.; and Pereira, F. (2005). Improving frame inerpolaion wih spaial moion smoohing for pixel domain disribued video coding. In 5 h EURASIP Conference on Speech and Image Processing, Mulimedia Communicaions and Services, Slovak Republic.

550 A. Elamin e al. 8. Aaron, A.; Rane, S.; Seon, E.; and Girod, B. (2004). Transform-domain Wyner-Ziv codec for video. Proceedings of SPIE Visual Communicaions and Image Processing, 520-528. 9. Bajcsy, J.; and Miran, P. (2001). Coding for he Slepian-Wolf problem wih urbo codes. In IEEE Global Telecommunicaions Conference, 2, 1400-1404. 10. Garcia-Frias, J. (2001). Compression of correlaed binary sources using urbo codes. In IEEE Communicaions Leers, 5(10), 417-419. 11. Bellifemine, F.; Capellino, A.; Chimieni, A.; Picco, R.; and Poni, R. (1992). Saisical analysis of he 2D- coefficiens of he differenial signal for images. Signal Process: Image Communicaion, 4(6), 477-488. 12. Avudainayagam, A.; Shea, J.M.; and Wu, D. (2008). Hyper-Trellis decoding of pixel-domain Wyner-Ziv video coding. IEEE Transacions on Circuis and Sysems for Video Technology, 18(5), 557 568. 13. Meyer, P.F.A.; Weserlaken, R.P.; Gunnewiek, R.K.; and Lagendijk, R.L. (2005). Disribued source coding of video wih non-saionary sideinformaion. Proceedings of Conference on Visual Communicaions and Image Processing, 857-866. 14. Weserlakena, R.P.; Gunnewiekb, R.K.; and Lagendijk, R.L. (2005). The role of he virual channel in disribued source coding of video. IEEE Inernaional Conference on Image Processing, 1, I-581-4. 15. Weserlaken, R.P.; Borcher, S.; Gunnewiek, R.K.; and Lagendijk, R. (2006). Finding a near opimal dependency channel model for a ldpc-based Wyner- Ziv video compression scheme. In 12 h annual conference of he Advanced School for Compuing and Imaging, 546-463, Lommel, Belgium. 16. Borade, S.; Nakiboglu, B.; and Zheng, L. (2008). Some fundamenal limis of unequal error proecion. IEEE Inernaional Symposium on In Informaion heory, ISIT 2008, 2222-2226. 17. Bries, C.; and Pereira, F. (2008). Correlaion noise modeling for efficien pixel and ransform domain Wyner-Ziv video coding. IEEE ransacions on Circui and Sysems for Video Technology, 18(9), 1177-1190. 18. Chien, W.J.; Karam, L.J.; and Abousleman, G.P. (2007). Block-adapive Wyner-Ziv coding for ransform-domain disribued video coding. In 32 nd IEEE Inernaional Conference on Acousics, Speech, and Signal Processing, I-525 I-528. 19. Xue, Z. (2009). Research and developmens of disribued video coding. PhD Thesis, Brunel Universiy, School of Engineering and Design. 20. Ince, S.; and Konrad, J. (2005). Geomeery-based esimaion of occlusion from video frame pairs. In IEEE Inernaional Conference on Acousics, Speech and Signal Processing, 2, ii/933 - ii/936, Philadelphia, PA, USA. 21. Lim, K.P.; Das, A.; and Chong, M.N. (2002). Esimaion of occlusion and dense moion fields in a bidirecional Bayesian framework. IEEE Transacions on Paern Analysis and Machine Inelligence, 24(5), 712-718. 22. Thoma, R.; and Bierling, M. (1989). Moion compensaing inerpolaion considering covered and uncovered background. Signal Processing: Image Communicaion, 1, 191 212.