Error resilience and recovery in streaming ofembedded video

Transcription

1 Signal Proessing 82 (2002) Error resiliene an reovery in streaming ofembee vieo Sungae Cho, William A. Pearlman Center for Next Generation Vieo Researh, Rensselaer Polytehni Institute, 110 Eighth Street, Troy, NY ,USA Reeive 1 September 2001 Abstrat The three-imensional (3-D) SPIHT oer is a salable or embee oer that has prove its eieny an its real-time apability in ompression ofvieo. A forwar-error-orreting (FEC) hannel (RCPC) oe ombine with a single automati repeat request (ARQ) prove to be an eetive means for proteting the bitstream. There were two problems with this sheme: the noiseless reverse hannel ARQ may not be feasible in pratie; an, in the absene of hannel oing an ARQ, the eoe sequene was hopelessly orrupte even for relatively lean hannels. In this paper, we introue a new metho ofpartitioning wavelet oeients into spatio-temporal (s-t) tree bloks to ahieve error resiliene. Eah ofthese s-t bloks orrespons to the full 3-D image region, beause roots of these trees are wavelet oeients taken at xe intervals in the root low-frequeny subban. Previously, we reporte on grouping ontiguous root subban oeients to generate s-t tree bloks that orrespon to loal 3-D regions. The new proeure brings higher error resiliene, sine lost oeients an be oneale with the surrouning oeients even ifsome ofthe oe s-t bloks are totally missing. The bitstreams ofthe oe s-t bloks are paketize an enoe with a hannel oe to orret errors an to prevent eoing oferroneous ata after errors are etete. Beause the separately enoe s-t bloks proue embee bitstreams, the pakets from the bitstreams are interleave to generate an embee omposite bitstream. The embee property, whereby suessive ompresse bits onvey suessively smaller value information, suggests unequal error protetion, where earlier bits are more strongly protete by the hannel oe than later bits. Therefore, unequal error protetion is also inorporate into our vieo bitstreams to bring an even higher egree ofresiliene to hannel bit errors. Our laims are supporte by extensive simulations with eoing ofthe various 3-D SPIHT bitstreams ompare to eah other an to MPEG-2. Superiority to MPEG-2 in noiseless an noisy hannels, uner equal onitions with or without FEC, is learly emonstrate by the results ofthese simulations.? 2002 Publishe by Elsevier Siene B.V. Keywors: SPIHT; Error resilient vieo oing; Unequal error protetion; Vieo ompression; Embee bitstream 1. Introution Wavelet zerotree image oing tehniques were evelope by Shapiro (EZW) [16], an further Corresponing author. aress: pearlman@ese.rpi.eu (W.A. Pearlman). evelope by Sai an Pearlman (SPIHT) [14], an have provie unpreeente high performane in image ompression with low omplexity. Later, Kim et al. extene the iea to the three-imensional SPIHT (3-D SPIHT) algorithm [10,11] that employe a temporal wavelet transform instea of motion ompensation, an ompare the result with the vieo ompression algorithms whih are generally use /02/$ - see front matter? 2002 Publishe by Elsevier Siene B.V. PII: S (02)

2 1546 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) nowaays, suh as MPEG-2 an H.263. They showe promise ofa very eetive an omputationally simple vieo oing, an also obtaine exellent results numerially an visually. However, wavelet zerotree oing algorithms are, like all algorithms prouing variable length oewors, extremely sensitive to bit errors. A single-bit transmission error may lea to loss ofsynhronization between enoer an eoer exeution paths, whih woul lea to a total ollapse ofeoe vieo quality. In a SPIHT ompresse bitstream, as will be explaine, a single error in a so-alle signiane map bit auses misinterpretation of all the sueeing bits. Beause ofthe progressive harateristi (embeeness) ofthe SPIHT bitstream, an early eoing error (often alle eoing failure for onvolutional oes) is partiularly isastrous. To ahieve robust vieo over noisy hannels, the work ofsherwoo an Zeger [17,18] for the SPIHT algorithm was extene to progressive vieo oing by Kim et al. They have shown in Refs. [12,19] that the robustness is inrease when we asae the 3-D SPIHT oer with rate-ompatible punture onvolutional (RCPC) hannel oer [9] an automati repeat request (ARQ) to protet the 3-D SPIHT bitstream from being orrupte by hannel errors. This approah inreases robustness, but is still suseptible to early eoing failure. Another approah towar proteting vieo ata from hannel bit errors is to moify the 3-D SPIHT algorithm to work inepenently in a number of so-alle spatio-temporal (s-t) bloks that are ivie into xe-length pakets an interleave to eliver a elity embee output bitstream. This algorithm is alle spatio-temporal tree preserving 3-D SPIHT (STTP-SPIHT) [4]. As a result, any bit error in the bitstream belonging to any one blok oes not aet any other blok, so that higher error resiliene against hannel bit errors is ahieve. Therefore, any early eoing failure aets the full extent of the GOF in the normal 3-D SPIHT, but in the STTP-SPIHT, the failure allows reonstrution of the assoiate region with lower resolution only. This algorithm gives exellent result in most ases, but may still experiene very early eoing errors, resulting in lower resolution vieo in spei regions. This metho was inspire by Refs. [5 7] in the el of image oing with the EZW algorithm [16]. In Ref. [2], Alatan et al. showe that the embee image bitstreams an be elivere with error resiliene maintaine by iviing the bitstreams into three lasses. They protet the sublasses with ierent hannel oing rates ofthe RCPC oer [9], an improve the overall performane against hannel bit errors. In this paper, we use the iea in [4], but a ierent metho for partitioning the wavelet oeients into s-t blok to solve those problems. Instea ofgrouping ajaent oeients, we group oeients at a xe interval in the lowest subban, epening on the number ofs-t bloks S. Then we trak the s-t relate trees ofthe oeients, an merge them together. As a result, the s-t bloks ofthe STTP-SPIHT orrespon to ertain loal regions, but the s-t bloks ofour new grouping metho orrespon to the full group of frames (GOF) with lower resolution. This grouping metho supports error onealment oflost oeients using surrouning oeients in the event ofeoing failure. We all this algorithm error resilient an error onealment 3-D SPIHT (ERC-SPIHT) algorithm. As with STTP-SPIHT, we separate the subbitstreams into xe length pakets, interleave them to obtain an embee omposite bitstream, an enoe them with a RCPC error-orretion oe with yli reunany hek (CRC). This kin ofhannel oe not only orrets errors, but also allows etetion ofeoing failures, so that eoing an ease in substreams where eoing failures our. Beause the subbitstreams are embee, the orretly reeive bits in eah subbitstream an be eoe to provie a reonstrution at lower resolution or auray. Then, we also show how the 3-D SPIHT enoe vieo bitstreams an be implemente with unequal error protetion by subiviing the embee bitstreams, prouing a hybri oer whih ombines the ERC-SPIHT algorithm an unequal error protetion. This metho an protet against early eoing error with high probability, beause we protet more strongly the beginning portion ofthe bitstream. The organization ofthis paper is as follows: Setion 2 shows error resilient 3-D SPIHT algorithm. Setion 3 shows unequal error protetion (UEP) ofthe 3-D SPIHT algorithm. Setion 4 provies simulation results. Setion 5 onlues this paper.

3 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) a a a b b b Fig. 1. Struture ofthe s-t relation of3-d SPIHT ompression algorithm. 2. Error resilient 3-D SPIHT algorithm 2.1. Bakgroun We begin by explaining briey the tree struture ofthe wavelet oeients an the SPIHT oing algorithm that operates on this struture. Fig. 1 shows how oeients in a 3-D transform are relate aoring to their spatial an temporal omains. Charater a represents a root blok ofpixels (2 2 2), an haraters b,, enote its suessive ospring progressing through the ierent spatial sales an numbers 1, 2, 3 label members ofthe same s-t tree linking suessive generations ofesenants. We use 16 frames in a GOF, therefore we have 16 ierent frames of wavelet oeients. We an observe that these frames not only have spatial similarity insie eah one ofthem aross the ierent sales, but also temporal similarity between frames. The SPIHT algorithm initially searhes the lowest s-t subban for the so-alle signiant oeients, whose magnitue is no less than a given threshol. The algorithm then searhes the trees roote in the lowest s-t subban for signiant oeients, an, in so oing, ns sets ofoeients whose magnitues are less than the threshol (i.e., insigniant sets) by a single binary eision that is sent to the bitstream. The tree noe that is the root ofan insigniant set is 3 put onto a list ofinsigniant sets (LIS). Whenever single oeients are foun to be insigniant, a 0 is sent to the bitstream an the loation ofthe oeient enters another list alle the list ofinsigniant points (LIP). When a oeient signiant for the threshol is foun, that ning is sent to the bitstream via a 1 along with its sign bit an its loation is put onto a list ofsigniant oeients (LSP). After the algorithm traverses the root subban testing all suh trees in this way, the threshol is halve an the proess is repeate rst by testing for signiane at the lowere threshol ofall oeients in the LIP an then for all sets in the LIS. Those oeients on the LSP at the previous higher threshol are rene in magnitue by sening their lower orer magnitue bits in the bit plane (binary expansion) orresponing to the urrent threshol. The proess ontinues through suessive halving ofthe threshol, until the bit buget is exhauste. The eoer mimis the enoer s exeution path, sine it reeives the signiane eision bits whih esribe it. The SPIHT bitstream omprises three kins of bits: signiane eision bits for single points or sets (alle signiane map bits); sign bits; an re- nement bits. Iferrors our in reeption ofsign or renement bits, only the assoiate oeients are reonstrute with value inauraies. On the other han, ifa signiane map bit is in error, then the eoing algorithm eviates from the enoer s exeution path an reonstruts the rest ofthe bitstream ompletely in error. In Ref. [4], we reporte the STTP-SPIHT ompression algorithm. Fig. 2 shows the struture an the basi iea ofthe STTP-SPIHT ompression algorithm. The STTP-SPIHT algorithm ivies the 3-D wavelet oeients into some number S ofierent groups aoring to their spatial an temporal relationships, an then to enoe eah group inepenently using the 3-D SPIHT algorithm, so that S inepenent embee 3-D SPIHT substreams are reate. These bitstreams are then interleave in bloks. Therefore, the nal STTP-SPIHT bitstream will be embee or progressive in elity, but to a oarser egree than the normal SPIHT bitstream. In this gure, we show an example ofseparating the 3-D wavelet transform oeients into four inepenent groups, enote by a, b,,, eah one ofwhih retains the s-t tree struture ofnormal 3-D

4 1548 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) a b a b aabbaaaabbbb aabbaaaabbbb a b a b aaaabbbb aaaabbbb aabbaabb aabbaabb aaaabbbbaaaabbbb aaaabbb bb aaaabbbb aaaabbb a a a a bbbb aaaabbbbaaaabbbb aa aaaaaa aa aaaaaa aa aaaaaa aa aaaaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa aaaa b b b b bb bbbb bb bb bb bb bbbb bbbb bbbbbb bbbb bbbb bbbbbbbb bbbbb bb b bbbb bbbb 3D SPIHT 3D SPIHT 3D SPIHT 3D SPIHT Enoer Enoer Enoer Enoer Stream 1 Stream 2 Stream 3 Stream 4 Blok Interleaver STTP-SPIHT bit stream Fig. 2. Struture ofthe STTP-SPIHT ompression algorithm. SPIHT [10,11], an these trees orrespon to the spei regions ofthe image sequenes. The s-t blok, whih is enote by a, mathes the top-left portion in all frames of the sequene transform. The other s-t bloks orrespon to the top-right, bottom-left, bottom-right frations of the image sequenes, an those s-t bloks are enote by b,,, respetively. The normal 3-D SPIHT algorithm is just a ase of S = 1, an we an exibly hoose S. When we hoose the number ofsubstream S for the image size of X Y, eah of X an Y shoul be ivisible by 2 L+1, where L is the number ofeomposition levels. If the axis is not ivisible by the number, we shoul exten the original image to be ivisible. For the Football sequene with 3 levels ofeomposition, we an hoose ertain values of S from 1 up to Propose metho STTP-SPIHT gave us exellent results in both noisy an noiseless hannel onitions while preserving all the esirable properties ofthe 3-D SPIHT [4]. However, this metho is also suseptible to early eoing error, an this error results in one or more small regions with lower resolution than the surrouning area. Sometimes, this artifat ours in an important region. To avoi this, early eoing error shoul be prevente so as to guarantee a minimum quality ofthe whole region. We an use a ierent metho for partitioning the wavelet oeients into s-t bloks to solve the problem. The 3-D SPIHT ompression kernel is inepenently applie to eah tree omprise ofthe wavelet oeients in the lowest subban an the spatially

5 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) Fig. 3. Graphial illustration ofthe two methos (STTP-SPIHT an ERC-SPIHT). relate oeients in the higher frequeny subbans. The algorithm proues sign, loation an renement information for the trees in eah pass. Therefore, we nee to keep the s-t relate trees to maintain ompression eieny ofthe 3-D SPIHT algorithm. However, we o not have to keep together the ontiguous wavelet oeients in the lowest subban, sine the kernel is inepenently applie to eah tree roote in a single lowest subban oeients an branhing into the higher frequeny subbans at the same s-t orientation. In our propose algorithm, therefore, we group the lowest subban oeients at some xe interval instea ofgrouping ajaent oeients. This interval is etermine by the number ofs-t bloks S, the image imensions, an number ofeomposition levels. Then, we trak the s-t relate trees ofthe oef- ients, an merge them together. Fig. 3 graphially ompares the two methos. The main avantage ofthe ERC-SPIHT is maintaining error resiliene with oing eieny. We also assign the same xe rates to eah substream. However, all ofthe subbloks ontain similar information about eah other, sine eah ofthe subbloks is ompose ofthe oeients not from a spei region, but from the whole region. Therefore, the xe assignments of bitrates make more sense to our propose metho. Another nie feature of the ERC-SPIHT is that the very early eoing failure aets the whole region beause the eoe oeients woul be sprea out to the whole area along with the sequene, an the oeients are oneale by the other surrouning oeients whih are eoe at a higher rate. When the eoing failure ours in the same position, the quality oferc-spiht is muh better than that ofsttp-spiht in visually an numerially (peak signal-to-noise ratio (PSNR)) beause ERC-SPIHT algorithm itselfhas the funtion oferror onealment. Therefore, the ERC-SPIHT no longer suers from small areas whih are eoe with a very low resolution. Fig. 4 shows the reovery apability oferc-spiht in a worst-ase example ofeoing failure. We use Football an Susie sequene, an oe at 1:0 bit=pixel with ERC-SPIHT (S = 16). We assume the eoing error ourre in the beginning of the substream number 2 (seon paket) for the Football sequene an the substream number 7 (seventh paket) for the Susie sequene, so that one of the substreams is totally missing. As a result, all ofthe wavelet oeients whih orrespon to the missing substreams are set to zeros. When the inverse wavelet transform is applie to the eoer, the orresponing regions are lle with blak pixels, beause the eoe pixel values are zeros. In this gure, (a) an () show the results from the ERC-SPIHT without error onealment, an (b) an () show them with error onealment. In this ase, we just use the average values ofsurrouning oeients for the missing oeients only in the root subban. As we an see, in the ase without error onealment, there are many blak spots in the images. However, when we use onealment for the missing oeients, we an reover the missing areas very well. 3. UEP of embee bitstreams 3.1. UEP of the 3-D SPIHT The 3-D SPIHT ompression kernel is ompose of two passes, a sorting pass an a renement pass, repeately performe until the total bits proue meet the bit buget. From the sorting pass, sign bits an loation bits are proue, an from the renement pass, renement bits are generate. The loation bits are results ofsigniane tests on sets ofpixels, inluing singleton sets, an omprise what is often alle the signiane map. As Alatan et al. i in Ref. [2], we lassify the bits into two lasses aoring to their bit error sensitivities. The sign bits an renement bits an be lassie as sign an renement bits (SRB), an the loation bits (LOB) an be lassie by themselves.

6 1550 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) Fig. 4. Coe to 1:0 bit=pixel, with ERC-SPIHT (P = 16): (a) top-left: Football sequene, without onealment (frame 1), PSNR = 17:96 B; (b) top-right: Football sequene, with onealment (frame 1), PSNR = 29:74 B; () seon row-left: original Susie sequene (frame 16) without onealment after the seventh substream is missing, PSNR = 19:61 B; an () Seon row-right: with onealment (frame 16), PSNR = 33:83 B. Ifany bit error ours in LOB, then the ompresse bit stream is useless after the point where the bit error ours. However, any bits in the SRB whih are affete by hannel bit errors o not propagate as long as the LOBs are error free. From our experimental results, the size ofthe SRB ranges from 20% to 25% ofthe original bitstream, epening on the rate. Fig. 5 shows bit error sensitivity ofa ompresse bitstream enoe by the 3-D SPIHT algorithm only with no subsequent arithmeti oing. In this gure, we ompresse the rst 16 frames of the monohrome Football sequene, an put 1 bit error to the ompresse bitstream from beginning to the en ofthe bitstream. In this gure, the x-axis represents the bit error position, an the y-axis means the average PSNR ofeoe sequene. As we an see, only one bit error in the beginning ofthe bitstream aets the whole bitstream, an makes the bitstream meaningless. In aition to that, we an see in (b) () many positions where the average PSNRs are very high ompare to the other positions. This lass is alle SRB, sine these bits represent sign an renement information of wavelet oeients. The other lass is also alle LOB, beause these bits ontain the information ofloation ofthe wavelet oeients an shoul be synhronize between enoer an eoer. In aition, the 3-D SPIHT algorithm has an important property that all the ompresse bits are positione in the orer oftheir ontribution to value. This means that SPIHT proues a purely embee or progressive bitstream, meaning that the later bits in the bitstream rene earlier bits, an the earlier bits are neee for the later bits to be useful. Fig. 6 represents average PSNR values versus bit-rates for the monohrome Football sequene. From this gure, we an see that the average PSNR value very rapily inreases when bit-rates are lower than 0:05 bpp, an most ofthe bits in this bit-rate are LOBs. Above this rate, PSNR inreases at muh more graually with bit-rate. This result implies that the very beginning ofthe bitstream shoul be more strongly protete against hannel bit errors than later portions ofthe bitstream. For this reason, even ifwe have only the beginning part ofthe bitstream, we an still get a rough renition ofthe soure. But ifwe lose just a small portion at the beginning part ofthe bitstream ontaining LOB bits, then we annot reonstrut anything from the bitstream. From this iea, we an further subpartition the

7 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) (b) () () Average PSNR Average PSNR Bit Error Position x Bit Error Position Average PSNR Average PSNR Bit Error Position x Bit Error Position x 10 4 Fig. 5. Average PSNR versus bit error position to represent bit error sensitivity ofthe 3-D SPIHT ompresse bitstream of monohrome Football sequenes (rst 16 frames) oe 1:0 bpp (bit per pixel): (a) top-left: average PSNRs of 70; 000 bit; (b) top-right: average PSNRs for the rst sorting pass of 1000 bit sample; () bottom-left: average PSNRs for seon renement pass; an () bottom-right: average PSNRs for the thir renement pass. LOB lass into two lasses LOB-a an LOB-b as Alatan et al. i in Ref. [2]. Eah lass orrespons to the earlier an later parts, respetively, ofthe bitstream. We an utilize these fats to get higher error resiliene against hannel bit errors. We separate the SRB an LOB in the original bitstream, an transmit the SRB rst with lowest error protetion (highest hannel oe rate), then LOB-a an LOB-b, eah with stronger protetion (lower hannel oe rate) than SRB, but with LOB-a reeiving a lower oer rate (higher protetion) than LOB-b. The reason to sen SRB bits rst is that the eoer nees sign bits one LOB bits iniating signiane are enountere. Figs. 7 an 8 graphially illustrate this iea. Fig. 7 shows the struture ofthe unequal error protetion 3-D SPIHT (UEP-SPIHT) an how the bits are lassie an ombine together. As we an see, we o not use the arithmeti oing for SRB bits to avoi error propagation among the bits. Fig. 8 presents the bit-rate assignments aoring to their bit error sensitivities an importane. As we an see in this gure, LOB-a shoul be highly protete, beause these bits are more important than others in terms ofbit sensitivities an the orer ofimportane, then LOB-b an

8 1552 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) Average PSNR Bit Rate Fig. 6. Average PSNRs versus bit-rates for monohrome Football sequenes. Fig. 7. UEP ofthe 3-D SPIHT algorithm. SRB an be protete with suessively higher hannel oing rates. As epite in Fig. 8, we sen the SRB bits rst, then sen the LOB bits. This means that while sening SRB bits, this bitstream is not progressive. However, after sening SRB bits, this bitstream is purely progressive, sine all the SRB bits are store in a buer, an the sign bits in this buer are aesse when LOB signiane bits are enountere. As we mentione before, the SRB segment ranges from 20% to 25% ofthe total bitstream for soure oe rates about 1 bpp (2:53 Mbps). The SRB size is relatively

9 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) Fig. 8. Bit rate assignment ofthe UEP. Fig. 9. Bitstream ofsttp-spiht algorithm. smaller at smaller bit-rates. Therefore, we get higher error resiliene against hannel bit errors while we sarie the progressiveness to a small extent. In the UEP-SPIHT heaer, we nee just one negligible aitional item ofinformation, the SRB size UEP with the ERC-SPIHT ERC-SPIHT gives exellent results in both noisy an noiseless hannel onitions while preserving all the esirable properties ofthe 3-D SPIHT. These results are shown later in Table 2. However, this metho still stops the eoing proess for the substream wherein the rst eoing error ours. When suh a eoing error ours, we must isar the following bits, but an eetively oneal the aete region. Furthermore, the higher protetion ofthe early part of the bitstream in the unequal error protetion sheme makes the potentially isastrous early eoing error muh less likely to our. To implement unequal error protetion to ERC- SPIHT, we partition the subbitstreams aoring to their bit sensitivities an the orer ofimportane. Fig. 9 shows us this iea. Every substream is ivie into SRB an LOB segments, enote by SRB1-SRBn, an LOB1-LOBn, where eah is ivie into its LOB-a an LOB-b segments. As we use in the 3-D SPIHT algorithm, we sen all the SRBs rst, an then sen LOBs. In orer to restore progressiveness to the omposite bitstream, we use a paket interleaving=einterleaving sheme for the LOB area to maintain progressiveness. The overhea ofthis metho is the information bits whih are save in eah subbitstream heaer to onvey its SRB size. The eision ofnumber ofpakets for eah lass is explaine in a later setion. To eoe the bitstream, the eoer reas the heaer rst, an put the SRBs to buer areas aoring to the information of the SRBs size as the bits are arriving. One all the SRBs have arrive, the eoer einterleaves the LOBs aoring to the paket size, sine the LOBs are sent as an interleave bitstream, an eoes the bitstreams with SRBs together. Therefore, early portions of this bitstream are protete strongly with little loss ofprogressiveness Soure=hannel oing rate Fig. 10 shows the system esription of3d=erc- SPIHT with RCPC oer. The funtions of paket interleaving an einterleaving are neee for the ERC-SPIHT an STTP-SPIHT, but not for regular 3-D SPIHT. Before RCPC enoing, we partition the bitstream into equal length segments of N bits. Eah segment of N bits is then passe through a CRC [3,13] parity heker to generate parity bits. In a CRC, binary sequenes are assoiate with polynomials of a ertain polynomial g(x) alle the generator polynomial. Hene, the generator polynomial etermines the error ontrol properties ofa CRC. Next, m bits, where m is the memory size ofthe onvolutional oer, are pae at the en ofeah N + + m bits ofthe segment an passe through the rate r RCPC hannel oer, whih is a type of punture onvolutional oer with the ae feature ofrate ompatibility. The eetive soure oing rate R e for the original 3-D SPIHT is given by R e = Nr N + + m R total; (1) where the unit of R e an R total an be either bits=pixel, bits=s, or the length ofbitstream in bits. The total number ofpakets M is alulate by R e =N, where R e is the bitstream length. In the ase ofunequal error

10 1554 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) D/ERC- Blok SPIHT Interleaving CRC+RCPC Noisy Channel Viterbi Deoer + CRC Chek Deinterleaving ERC- SPIHT return hannel (optional) Fig D=ERC-SPIHT with RCPC system framework. Table 1 Relative perentages ofsoure=hannel (RCPC) bits at total transmission rate of2:53 Mbps when BER is 0.01 r Soure (%) Channel (%) 3-D SPIHT 2= SRB 4= UEP-SPIHT LOB-a 4= LOB-b 2= protetion, we rst get the R e SRB an M SRB aoring to Eq. (1). Then, R e an R LOB-a e LOB-b an be alulate by r LOB-a N + + m R LOB-a + r LOB-b N + + m R LOB-b =M M SRB ; (2) where R LOB-a + R LOB-b = R total R SRB. Therefore, we an get R LOB-a an R LOB-b. Table 1 shows the relative perentage ofsoure oing bits an hannel oing bits using this equation for monohrome Football sequenes at total transmission rate of2:53 Mbps when the bit error rate (BER) is In this table, we assigne the same bit buget to the normal 3-D SPIHT as in [12,19] aoring to the hannel oing rate. For the ERC-SPIHT with unequal error protetion, we use the same bit buget as in the ase ofnormal 3-D SPIHT. We rst assign same hannel oing rate to the LOB-b (r LOB-b), then assign one level lower rate to the LOB-a (r LOB-a), an one level higher rate to the SRB (r SRB ). Then, (R LOB-a), (R LOB-b), an (R SRB ) are alulate by Eqs. (1) an (2). 4. Results In our test oferror resiliene, we assume that the hannel is binary symmetri (BSC). For MPEG-2, we use 15 frames in a group of pitures (GOP), an the I=P frame istane is 3 (IBBPBBP :::). For the 3-D=ERC-SPIHT, 16 frames in a GOF are use, an a yai three-level transform using 9=7 biorthogonal wavelet lter [1] is applie to the image sequenes. For the 3-D SPIHT an MPEG-2, we sen the bitstream sequentially in 200 bit pakets, an for the ERC-SPIHT, we interleave the streams in 200 bit pakets to maintain embeeness, an the reeiver einterleaves the bitstream to a series ofsubstreams, eah one ofwhih is eoe inepenently. The algorithm is then teste using the monohrome Football sequenes. The istortion is measure by the PSNR: ( ) PSNR = 10 log 10 B; (3) MSE where MSE enotes the mean square error between the original an reonstrute image sequenes. All PSNRs reporte for noisy hannels are averages over 50 inepenent runs. Fig. 11 represents the frame by frame omparison ofpsnrs of Football an Susie sequenes oe with 1:0 bit=pixel (2:53 Mbps) without bit errors an forwar error orreting. The soli line on top shows the PSNR values ofnormal 3-D SPIHT with Susie sequene, an the seon soli line means PSNR value with Football sequene. The ashe ot lines represent ERC-SPIHT with S = 16, an ashe lines mean STTP-SPIHT with S = 16, an otte lines show the MPEG-2 oe sequene. As we an see, there are just small losses (0.1 0:4 B) with ERC-SPIHT with partitioning to 16 subgroups, an this urve follows the normal 3-D SPIHT very losely. In the Susie sequene, the PSNRs in STTP-SPIHT with S =16 is about 1 B worse than those ofthe normal 3-D SPIHT in every frame. This ierene of PSNRs is mainly ue to the ineient rate alloation among

11 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) Normal SPIHT ERC SPIHT (S=16) STTP SPIHT (S=16) MPEG 2 Susie 45 PSNR (B) Football Frame Number Fig. 11. Comparison of frame by frame PSNR (B) of the Football an Susie sequenes oe to 1:0 bit=pixel with normal 3-D SPIHT, ERC-SPIHT (S = 16), STTP-SPIHT (S = 16), an MPEG-2 without hannel bit errors. Table 2 Comparison ofaverage PSNRs among 3-D SPIHT, ERC-SPIHT (S =4; 10; 16; 55), STTP-SPIHT (S =4; 10; 16; 55), an MPEG-2 at total transmission rate of2:53 Mbps in noiseless hannel Football Susie MPEG D SPIHT STTP- ERC-SPIHT STTP- ERC- SPIHT SPIHT SPIHT SPIHT S = S = S = S = substreams, beause the Susie sequene ontains a large portion ofstill bakgroun extening to the full GOF. The rates of these substreams shoul be alloate unequally to reet the unequal amount ofativity in ierent substream regions. Therefore, ERC-SPIHT with S = 16 suessfully follows the PSNR values ofthe normal 3-D SPIHT with just small PSNR ierenes. Table 2 shows the omparison ofaverage PSNRs among 3-D SPIHT, ERC-SPIHT (S =4; 10; 16; 55), STTP-SPIHT (S =4; 10; 16; 55), an MPEG-2 at total transmission rates of2:53 Mbps of Football an Susie sequenes with BER of0. As we an see in this table, the average PSNRs oferc-spiht with S = 16 are :13 B higher than those of the STTP-SPIHT with S = 16, an still :25 B higher than those ofthe STTP-SPIHT with S = 10. In our simulation oferror resilient vieo transmission with error orretion apability, all ofthe normal 3-D SPIHT, ERC-SPIHT, STTP-SPIHT an MPEG-2 bitstreams were protete ientially. As in previous works [12,17,19], we protete the 200 bit pakets with the CRC, = 16 bit parity hek while generator polynomial g(x) =X 16 + X 14 + X 12 + X 11 + X 8 + X 5 +X 4 +X 2 +1, an RCPC hannel oer with onstraint length m = 6. We fouse on BER of =0:01 an 0.001, beause the BERs ofmost wireless ommuniation hannels are =0: We set the total transmission rate R total to 2:53 Mbps; r=2=3 for =0:01 an 8=9 for =0:001 for the equal error protetion (EEP). In our ase ofuep, we use a ertain r only for LOB-b, an use the one level higher rate available to us for the SRB, an one level lower rate for the LOB-a with the same transmission rate of

12 1556 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) :53 Mbps. In the ase of =0:01, the RCPC rate for LOB-b = 2=3, an the rate for the SRB = 4=5, an the rate for the LOB-a = 4=7. One we eie the hannel oing rate, we an easily alulate the bit buget for the three lasses using Eqs. (1) an (2). At the estination, the Viterbi eoing algorithm [8,15] is use to onvert the pakets ofthe reeive bitstream into a 3-D=ERC-SPIHT an MPEG-2 bitstream. In the Viterbi algorithm, the best Path hosen is the one with the lowest path metri that also satises the heksum equations. In other wors, eah aniate trellis path is rst heke by omputing a = 16 bit CRC. When the hek bits iniate an error in the blok, the eoer usually xes it by ning the path with the next lowest metri. However, ifthe eoer fails to eoe the reeive paket within a ertain epth ofthe trellis, it stops eoing for that 3-D=ERC-SPIHT stream. For MPEG-2, whih is not embee an nees the full bitstream to see the whole frames, when eoing failure ours, we an use one oftwo shemes. One is just to use the orrupte paket, an the other is to put all 0 s to the orrupte paket. In this paper, we use the orrupte paket itself. Fig. 12 shows the omparison of Football sequene with RCPC an BER = 0:01. In this gure, (a) is the original Football sequene (frame 15). Typial reonstrutions of Football sequene at total transmission rate of2:53 Mbps an hannel BER of0.01 with S = 16 an MPEG-2 are shown in Figs. 12(b) (). As we an see, in Fig. 12(b), the ERC-SPIHT stops eoing for the substream where eoing failure ours. Therefore, any early eoing failure aets the whole region, an the oeients are oneale by the other surrouning oeients whih are eoe at higher rates. In this example, the very early eoing error ours in the 10th substream, but it is very har to isern the aete region. However, the MPEG-2 eoe sequene in Fig. 12(), the eoing failure aets some blok, an the blok is lle with some other piture s blok. In the ase ofunequal error protetion ERC-SPIHT, the probability ofvery early eoing error is very low beause this metho strongly protets the earlier part ofthe bitstream, an usually gives a nie result. Fig. 12() shows a typial example ofa frame in the eoe sequene (frame 15). Table 3 shows the omparison ofaverage PSNRs among 3-D SPIHT, UEP-SPIHT, an Table 3 Comparison ofaverage PSNR in B over 50 inepenent trials among 3-D SPIHT, UEP-SPIHT, an UEP=ERC-SPIHT at total transmission rate of2:53 Mbps BER EEP 3D-SPIHT=RCPC EEP ERC-SPIHT=RCPC UEP 3-D SPIHT=RCPC UEP ERC-SPIHT=RCPC MPEG=RCPC UEP=ERC-SPIHT for monohrome Football sequene at total transmission rate of2:53 Mbps. As we an see, the average PSNR ofuep-spiht is about 2 5 B higher than those ofthe equal error protetion 3-D SPIHT (EEP-SPIHT). In the hybri metho, when BER is 0.01 the average PSNR is 30:03 B, whih is a little higher than that ofequal error protetion oferc-spiht, an when BER is the average PSNR is 31:75 B. In aition to the higher PSNR, we have prevente very early eoing error using the lower hannel oing rate for the LOB-a. The merit over the unequal error-protetion of3-d SPIHT is that we an get onsistent results. In other wors, the unequal error-protete 3-D SPIHT still stops eoing whenever eoing failure ours, however, the unequal error protete ERC-SPIHT prevents very early eoing error eetively, an still operates until S eoing failures our. 5. Conlusions We have introue a new partitioning metho of wavelet oeients to support error resiliene an error onealment (ERC-SPIHT). The results show that this ierent metho ofgrouping wavelet oeient tree roots at a xe interval an enoing these interleave tree bloks inepenently provies high egrees oferror resiliene, error reovery, an oing eieny. We also integrate UEP into the 3-D SPIHT an the ERC-SPIHT ompresse bitstreams. The result is very promising while sariing progressiveness to a small egree. The most remarkable thing is that we eetively mitigate the early eoing error for the embee bitstream. In this paper, all our work is one with hannel bit errors over the binary symmetri hannel (BSC). In aition, our metho oul be use in paket erasure hannels, beause a eo-

13 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) Fig Football sequene (frame15): (a) top-left: original sequene; (b) top-right: using ERC-SPIHT (n = 16)=RCPC with BER = 0:01, PSNR = 29:64 B; () bottom-left: using MPEG-2=RCPC with BER = 0:01; PSNR = 27:45 B; an () bottom-right: using ERC-SPIHT with UEP with BER = 0:01; PSNR = 30:14 B. ing failure is equivalent in eet to a paket erasure in our sheme. For a bursty paket erasure hannel, sine subbitstreams are interleave by pakets, a burst ofpaket erasures or eoing failures woul ause termination ofeoing in a number ofsuessive subbitstreams. In that ase an in the ase ofhigher paket erasure rate, the number S ofsubbitstreams shoul inrease to minimize the eet ofpaket erasures on the reonstrute vieo. Further researh is neee to evaluate our methos an moify them, if neessary, for faing an paket erasure hannels. Referenes [1] M. Antonini, M. Barlau, P. Mathieu, I. Daubehies, Image oing using wavelet transform, IEEE Trans. Image Proess. 1 (1992) [2] A. Ayin Alatan, M. Zhao, A.N. Akansu, Unequal error protetion ofspiht enoe image bit streams, IEEE J. Selete Areas Commun. 18 (June 2000) [3] G. Castagnoli, J. Ganz, P. Graber, Optimum reunany hek oes with 16-bit, IEEE Trans. Commun. 38 (January 1990) [4] S. Cho, W.A. Pearlman, A full-feature error resilient salable vieo oe base on the Set Partitioning in Hierarhial Trees (SPIHT) algorithm, IEEE Trans. Ciruits Systems Vieo Tehnol. 12 (Marh 2002) [5] C.D. Creusere, A family of image ompression algorithms whih are robust to transmission errors, Pro. SPIE 2668 (January 1996) [6] C.D. Creusere, Robust image oing using the embee zerotree wavelet algorithm, Proeeings ofthe Data Compression Conferene, Marh 1996, p [7] C.D. Creusere, A new metho ofrobust image ompression base on the embee zerotree wavelet algorithm, IEEE Trans. Image Proess. 6 (10) (Otober 1997) [8] G.D. Forney Jr., The Viterbi algorithm, Pro. IEEE 61 (January 1994) [9] J. Hagenauer, Rate-ompatible punture onvolutional oes (RCPC oes) an their appliations, IEEE Trans. Commun. 36 (April 1988) [10] B.-J. Kim, W.A. Pearlman, An embee wavelet vieo oer using three-imensional set partitioning in hierarhial trees, Proeeings ofthe Data Compression Conferene, Marh 1997, pp [11] B.-J. Kim, Z. Xiong, W.A. Pearlman, Low bit-rate salable vieo oing with 3D set partitioning in hierarhial trees (3D SPIHT), IEEE Trans. Ciruits Systems Vieo Tehnol. 10 (Deember 2000) [12] B.-J. Kim, Z. Xiong, W.A. Pearlman, Y.S. Kim, Progressive vieo oing for noisy hannels, J. Visual Commun. Image Representation 10 (1999) [13] T.V. Ramabaran, S.S. Gaitone, A tutorial on CRC omputations, IEEE Miro 8 (August 1988) [14] A. Sai, W.A. Pearlman, A new, fast an eient image oe base on set partitioning in hierarhial trees, IEEE Trans. Ciruits Systems Vieo Tehnol. 6 (June 1996)

14 1558 S. Cho, W.A. Pearlman / Signal Proessing 82 (2002) [15] N. Seshari, C. Sunberg, List Viterbi eoing algorithm with appliations, IEEE Trans. Commun. 42 (1994) [16] J.M. Shapiro, Embee image oing using zerotrees ofwavelet oeient, IEEE Trans. Signal Proess. 41 (Deember 1993) [17] P.G. Sherwoo, K. Zeger, Progressive image oing on noisy hannels, Proeeings ofthe DCC, April 1997, pp [18] P.G. Sherwoo, K. Zeger, Progressive image oing for noisy hannels, IEEE Signal Proess. Lett. 4 (July 1997) [19] Z. Xiong, B.-J. Kim, W.A. Pearlman, Progressive vieo oing for noisy hannels, Proeeings of the IEEE International Conferene on Image Proessing (ICIP 98), Vol.1, Otober 1998, pp