Grouping and Retrieval Schemes for Stored MPEG. Video. Senthil Sengodan, Victor O. K. Li. University of Southern California

Transcription

1 Grouping and Retrieval Scheme for Stored MPEG Video Senthil Sengodan, Victor O. K. Li Communication Science Intitute Department of Electrical Engineering Univerity of Southern California Lo Angele, CA Abtract MPEG, in addition to being an international tandard, i currently the mot popular coding cheme for tored video. For everal application that require tored video, uch a videoon-demand, it i advantageou to group MPEG frame into egment. Retrieving egment intead of individual frame can reult in a more ecient ue of dik retrieval reource and conequently, in the upport of a larger number of uer and in a more cot-eective ytem. Depending on the value of parameter uch a deired uer quality of ervice, ize and cot of required buer, number of dik to tripe a egment acro, etc., a certain egment length (dened a the number of frame per egment) may be preferred over other. In thi paper, we propoe two grouping cheme - the genealized grouping and retrieval (GGR) cheme and the dicard-free grouping and retrieval (DGR) cheme - that may each be ued to achieve any deired egment length. Propertie of uch grouping cheme and formulae that may be ued to achieve uch grouping have been provided. Since a egment i an atomic unit of video retrieval, retrieval of ome egment i kipped during fat playback. In GGR, ome frame belonging to retrieved egment may be dicarded due to the unavailability of other frame neceary for their decoding. Tradeo between dierent kind of fat playback and the number of dicarded frame i dicued. DGR, on the other hand, avoid the dicard of any frame belonging to retrieved egment. The advantage and diadvantage of each cheme are lited. Keyword: MPEG, video-on-demand, video torage, video retrieval Primary Technical Subject Area: Multimedia Communication 1

2 1 Introduction 1.1 MPEG Coding Ecient torage and retrieval of video [4] i extremely important for the ucce of emerging application uch a Video-on-demand (VOD) [2, 3, 6, 5, 11, 12]. The mot popular method of coding for application that require tored video i the international tandard Motion Picture Expert Group (MPEG) coding [7, 8]. An MPEG coded video contain three dierent kind of frame - Intraframe (I), Predictive (P ) and Bidirectional (B). An I frame may be decoded independently; a P frame require the previou I=P frame for decoding; and, a B frame require the previou a well a the ubequent I=P frame for decoding. For intance, in Figure 1, frame 3 i needed to decode frame 6 while frame 6 and 9 are needed to decode frame 7 a well a 8. The ratio of the number of I, P and B frame i a coding parameter and may be varied depending on the deired video quality and torage ize. A indicated in Figure 1, we will denote the number of P frame between two conecutive I frame by (m? 1), and the number of B frame between two conecutive I=P frame by (q? 1). Each frame i aigned a frame number which depend on it poition in the playback order. The pattern indicated in Figure 1 repeat, i.e., frame number 21 i an I frame, frame number 22 i a B frame, and o on. 1.2 Video Segment A video egment i a group of frame and i conidered to be an atomic (indiviible) unit for retrieval purpoe. Since the average ize of a egment i larger than that of a frame, a egment i better uited for triping [9] acro a larger number of dik. Thi i becaue dik data i tored in block and triping one frame acro everal dik may reult in a torage of le than one block per dik. Thi mean that during retrieval the entire block i retrieved wherea only a mall portion of the retrieved data i ueful. Striping acro a larger number of dik ha the advantage that a larger retrieval bandwidth may be obtained. Since a dik head ha to eek the block in which the data to be retrieved i tored and ince dierent block toring portion of the ame egment are uually located on the ame dik track, retrieval of a larger amount of data (i.e., a egment intead of a frame) reult in a decreae in the average eek time for a unit of video data. Due to thee reaon, frame are often grouped into egment and a egment i taken to be the mallet (atomic) unit of video retrieval. While a larger egment ize ha the advantage of accommodating a larger number of uer in the ytem becaue of a more ecient ue of dik retrieval reource, it alo uer from certain drawback. A large egment ize require more buer pace (at the erver and/or at the uer' et-top box) which add to the cot of the ytem. Hence, a tradeo exit between the number of uer that can be upported and the required buer. Baed on the 2

3 Playback Order : I B B P B B P B B P B B P B B P B B P B B Frame Number : q=3 qm=21 Figure 1: MPEG frame relative weight of thee two parameter, a certain egment length (dened a the number of frame per egment) may be preferred. Hence, uitable and ecient grouping of frame for thi egment length i needed. 1.3 Frame/Segment Skipping During fat playback, a deirable and often ued approach i to kip certain frame and diplay other at the ame rate a that of normal playback. Thi i becaue the frame diplay rate (or refreh rate) of video monitor a well a TV creen i contant, i.e., frame can only be diplayed at a xed rate. Let u denote the normal playback rate by n frame per econd (fp) and the fat playback rate by n 1 fp where n 1 > n. Thi fat playback i imulated by diplaying n out of every n 1 frame in the playback order at the ame rate of frame diplay a normal playback. It i een that the choice of frame to be diplayed determine the nature and quality of fat playback diplay. We hall refer to a fat playback a being a piecewie continuou diplay of conecutive frame at the normal playback rate. Each piece in thi piecewie continuou diplay comprie conecutive frame in the playback order. During the period that a piece i played back, the portion of the video that comprie the piece i being viewed at the normal playback rate. Hence, in order to diplay n out of n 1 frame, the piecewie continuou playback may either comprie everal hort piece or a few long piece. In the cae of the former, the ize of the break between piece (i.e., the number of conecutive frame not diplayed) i hort while in the latter it i long. Depending on the deired fat playback appearance, the piece ize i determined. Thi i illutrated in Figure 2 where the fat playback rate (n 1 ) i twice the normal playback rate (n). The frame that would be diplayed for three piece ize (1, 2 and 5) i indicated. We jut aw that ome frame are not diplayed during fat playback. In order to decreae the load on the video erver a well a the network (i.e., in order to conerve dik retrieval a well a network reource), it i deirable that thee frame not be retrieved from the erver and tranported to the uer via the network. Since the atomic unit of retrieval i a video 3

4 Frame Number : Fat Playback : n 1 = 2n Piece Size Diplayed Frame Figure 2: Piecewie continuou diplay for fat playback egment, thi implie that retrieval of only certain egment take place from the video erver, i.e., retrieval of certain egment i kipped. It i alo deirable that all frame belonging to retrieved egment be diplayed. However, thi may not be poible becaue decoding of a frame may require other frame that were not retrieved. Conequently, grouping of frame into egment and the choice of egment to be retrieved for fat playback hould be uch that the number of retrieved frame that are not diplayed i minimized. 1.4 Reearch Motivation Although the advantage of grouping frame into egment ha been realized by everal reearcher, not much work ha been done on how uch a grouping can be done. In [1], a egment contitute all frame from one I frame to (and excluding) the next. The number of frame per egment, here, i qm and xed by the coding parameter q and m. Depending on the particular application (ay, VOD) and on the value of parameter uch a deired diplay quality, buer cot etc., a certain egment ize may be preferred over other. Thu, cloed form equation that directly indicate the frame to be grouped to attain thi egment ize are deirable. Similarly, cloed form equation that may be ued to eaily determine the egment to be retrieved during fat playback are needed. In thi paper, we propoe two cheme - a generalized grouping and retrieval (GGR) cheme and a dicard-free grouping and retrieval (DGR) cheme - for grouping frame into egment of any deired egment length. Cloed form equation are provided that may be ued to eaily determine the frame to be grouped together into a egment. Guideline are provided to determine the egment to be retrieved for any playback rate. Since frame belonging to retrieved egment may have to be dicarded in DGR due to the lack of other frame needed for their decoding, tradeo between dierent kind of fat playback (i.e., dierent piece ize) and the number of dicarded frame i dicued. 4

5 (a) Playback Order : I B B P B B P B B P B B P B B P B B P B B Frame Number (i) : (b) Minimal Caual Order : I P B B P B B P B B P B B P B B P B B I B B Poition ( i ) : Frame Number (i) : Minimal Caual Order: I B B P B B P B B P B B P B B P B B P B B I B B (with dummy frame) (c) Poition ( i ) : Frame Number (i) : 0 x x Figure 3: Minimal caual order The ret of the paper i organized a follow. Section 2 decribe GGR, while Section 3 decribe DGR. We conclude in Section 4. 2 GGR Scheme 2.1 Minimal Caual Order In Figure 1, we howed the frame playback order which ha been redrawn in Figure 3(a) for convenience. The playback order i not caual with repect to frame requirement for frame decoding. A frame in the playback order may require frame later in the order for it decoding. Conider the cae where frame are retrieved periodically and one at a time according to the playback order. The time period within which one frame i retrieved equal the time for playing back one frame. From Table 1, it i een that q frame need to be prefetched prior to the tart of playback in order to avoid video tarvation. Thi reult in a maximum buer occupancy of q frame. Since only the B frame require frame appearing after it in the playback order for decoding purpoe, moving every I=P frame in the playback order (q? 1) place up would reult in a caual order. Thi caual order i hown in Figure 3 and i referred to a the minimal caual order. Let frame be retrieved periodically and one at a time according to the minimal caual order, with the time period of frame retrieval being equal to the frame playback duration. A hown in Table 2, only two frame have to be prefetched prior to 5

6 Time Retrieved frame Played back frame # frame in Buer Table 1: Periodic frame retrieval according to playback order Time Retrieved frame Played back frame # frame in Buer Table 2: Periodic frame retrieval according to minimal caual order tart of playback in order to avoid video tarvation. A maximum buer occupancy of only two frame i required. Theorem 1: Let frame be retrieved one at a time and periodically with the retrieval period equaling the playback period. The number of frame that need to be prefetched prior to tart of playback (in order to avoid video tarvation) and conequently the maximum number of frame that need to be buered i minimized when the order of frame retrieval follow the minimal caual order. Proof: When frame are retrieved according to the minimal caual order, irrepective of the value of q and m, only two frame need to be prefetched prior to tart of playback. The maximum number of frame that need to be buered i two. Prefetching only one frame would not work for any order of frame retrieval becaue of the decoding requirement of B frame. When a B frame i to be diplayed, the I=P frame following it in the playback order need to be preent in the buer for the ucceful decoding of the B frame. Thi mean that we need at leat a buer torage of two frame. 2 Storage and retrieval of MPEG frame according to a caual order reult in a impler decoding of frame. The earlier that a certain frame appear in the caual order compared to it poition in the playback order, the longer it need to be buered. A frame whoe poition in the caual order i later than it poition in the playback order doe not have to be buered. If i = f0; 1; g denote the frame number and i = f0; 1; g the correponding poition of frame i in a caual order, then 6

7 max(i? i + p? 1) +, 8 i = f0; 1; g i an indicator of the maximum time between any frame retrieval and playback. Here, p denote the number of frame that have to be prefetched and (n) + = ( n; if n 0; 0 if n < 0: Theorem 2: Of all poible caual order, the minimal caual order minimize max(i? i + p? 1) +, 8 i = f0; 1; g. Proof: The requirement of the I=P frame following a B frame for the decoding of the B frame i what make the playback order non-caual. Moving each I=P frame in the playback order (q? 1) place ahead would reult in a caual order which i the minimal caual order. Since the minimal caual order wa obtained by the leat modication to the playback order, it minimize max(i? i ) +. Moreover, the minimal caual order ha p = 2 which i the mallet of all caual order. Hence, the reult. 2 From Figure 3(b), it i een that the rt I and the rt P frame appear conecutively in the minimal caual order. Any ubequent I=P frame i eparated from it neighboring I=P frame by (q? 1) B frame in the torage order. In order to make the tart of the torage order imilar to the ret, we introduce (q? 1) dummy B frame between the rt I and P frame. The frame number of thee frame have been denoted by x and the reulting order i hown in Figure 3(c). Similarly, any boundary eect at the end of the video may alo be reolved by introducing uitable dummy frame. Henceforth, we hall only deal with the caual order that include the dummy frame and refer to uch a caual order a the minimal caual order (intead of the one without the dummy frame). 2.2 Grouping We hall dene a frame torage order to be an arrangement of frame uing which the frame that belong to a particular egment can be directly determined. Let the egment length be frame per egment. In order to group frame into egment of length, every et of conecutive frame beginning at the tart of the frame torage order are grouped together. Due to the propertie enjoyed by the minimal caual order, we hall chooe the minimal caual order a the frame torage order. Jut like frame, each egment i aigned a egment number with the rt egment being aigned a number zero. We hall refer to the arrangement of egment in acending egment number a the egment torage order. It i eay to ee that the egment torage order i caual with regard to frame decoding whenever the frame torage order i caual. When the frame torage order i caual, a frame belonging 7

8 Storage Order : IB BP BB PB BP BB PB BP BB PB BI BB PB BP BB PB BP BB PB BP BB Frame Number : 0,x x,3 1,2 6,4 5,9 7,8 12,10 11,15 13,14 18,16 17,21 19,20 24,22 23,27 25,26 30,28 29,33 31,32 36,34 35,39 37,38 Segment Number : c = LCM (qm,) = 21 Figure 4: Grouping cheme to any egment doe not require a frame belonging to a egment later in the order (i.e., a egment with a larger egment number) for decoding purpoe. We dene a egment et to be a et compriing the mallet number (ay, c) of conecutive egment in the torage order that form a egment pattern. The rt egment et include c conecutive egment beginning with the rt egment of the egment torage order, i.e., egment number 0. Property G1: The number of egment in a egment et equal c = LCM (qm;). Proof: The minimum number of frame in the frame torage order that form a frame pattern i qm. Since et of conecutive frame in the frame torage order contitute a egment, the number of frame encountered before a egment pattern repeat i LCM(qm; ). Thu, the number of egment encountered before a egment pattern repeat i LCM (qm;) egment. 2 Thu, egment with number (cj + i) are imilar egment for all i = f0; 1; ; c? 1g and all nonnegative j, i.e., they contain the ame kind of frame (I, P or B) in the ame LCM (21;2) order. In Figure 4, for the value q = 3, m = 7, = 2, we have c = = 21. Segment 2 with number 1, 22, 43, are imilar egment, which i a BP egment in thi cae. Property G2: The frame contained in a egment with number i have the poition i; i + 1; i + 2; ; i +? 1 in the frame torage order. Proof: Thi immediately follow from the fact that every et of conecutive frame beginning at the tart of the frame torage order contitute a egment. 2 Property G3: A frame with poition j in the torage order ha a frame number (which i it poition in the playback order) of j? d j q? b j q ceq. Proof: The poition of an I=P frame in the torage order i the ame a that in the playback order (i.e., equal it frame number). Thi mean that if the frame with poition j in the 8

9 torage order i an I=P frame, then it' frame number hould equal j. A frame i an I=P frame if and only if it poition j in the torage order i a multiple of q. In uch a cae, the expreion above equal j ince the econd term goe to zero. The poition of a B frame in the playback order i (q? 1) place ahead of it poition in the frame torage order. Thi mean that if the frame with poition j in the torage order i a B frame, then it' frame number hould be (j? q). A frame i a B frame if and only if it poition j in the torage order i not a multiple of q. The term d j q? b j ce equal one whenever q j i not a multiple of q, and the expreion above equal (j? q). The non-exitent B frame (denoted by x) return a negative frame number (becaue they are the frame before the I frame with number 0) and can be eaily identied. 2 Uing the expreion in Propertie G2 and G3, given any egment number (i.e., the poition of any egment in the egment torage order), we can determine the frame number of each frame contained in the egment. With a knowledge of the frame number of each frame contained in each egment of any egment et, the frame number of the frame contained in any other egment may be eaily determined. Thi i becaue correponding egment in conecutive egment et contain frame whoe frame number are oet by c. We hall now dicu egment retrieval during fat playback. We are intereted in two parameter - the piece length and the percentage of retrieved frame that have to be dicarded. 2.3 Grouping propertie ueful for egment retrieval The normal playback rate i n fp and the fat playback rate i n 1 fp where n 1 > n. On an average, n out of every n 1 frame need to be diplayed. Canceling the highet common n factor (HCF) between the two, we need to diplay out of every n 1 frame. HCF (n;n 1 ) HCF (n;n 1 ) The atomic unit of retrieval i not a frame but a egment. If we know the egment to be retrieved out of the egment belonging to an integral number (ay, d) of conecutive egment et, then the egment to be retrieved in any egment et can be immediately determined. Thi i becaue the egment to be retrieved in egment et j are imilar to thoe retrieved in egment et (j mod d). The number d i the mallet integer uch that the ratio of the number of frame diplayed to the number of frame in d egment et equal n=n 1. Since a egment et contain c frame, d i choen uch that the number of frame in d egment et n equal LCM(c; 1 ). Hence, we need to diplay LCM(c; n 1 ) HCF (n;n 1 ) HCF (n;n n 1 ) n 1 out of every n LCM(c; 1 HCF (n;n 1 ) frame. We will denote the former number by a and the latter by b. We ) need to diplay a out of b frame. We have d = b=(c). In Figure 5, d row of egment are hown with each row containing egment belonging to the ame egment et. For the ret of thi dicuion, we will aume that a egment can contain at mot one I frame. Thi mean that the maximum egment ize equal qm. Extenion to the cae of a larger egment ize can be eaily done and i not dealt with here. 9

10 Property G4: Out of the c egment in a egment et, the number of egment that contain an I frame equal c qm. Proof: The number of frame in a egment et i c, and from the denition of a egment et, thi number i a multiple of qm. Since each et of qm frame contain exactly one I frame, the number of egment containing an I frame equal c qm. 2 Property G5: (a) Of all egment that contain an I frame in them, the l-th egment (in the egment torage order) can be repreented a l = i c +j for integer l; i = f0; 1; 2; g qm and j = f0; 1; ; c qm? 1g. (b) The egment number of the l-th uch egment i ic + bj qm c. (c) The poition of the I frame in the l-th uch egment i jqm? bj qm c. Proof: (a) Thi follow directly from Property G4. i denote the egment et that the l-th egment with an I frame in it belong to, while j i the number of egment in egment et i that are ahead of the concerned egment (in the egment torage order) and that have an I frame in them. (b) Since i denote the egment et containing the l-th egment with an I frame in it, the number of egment in previou egment et equal ic. bj qm c denote the number of egment in egment et i that are ahead of the concerned egment (in the egment torage order). (c) Since a egment et contain a et of egment that form a egment pattern in the egment torage order, the poition of the I frame in the l-th egment with an I frame in it doe not depend on the value of i. jqm denote the number of frame in egment et i that are ahead of the I frame of the concerned egment (in the frame torage order). bj qm c denote the number of frame in egment et i that are ahead of the I frame of the concerned egment (in the frame torage order) and which do not belong to the concerned egment. The dierence of thee two term give the number of frame ahead of the I frame (in the frame torage order) in the concerned egment. Thi i merely the poition of the I frame in the concerned egment. 2 Property G6: When a egment with an I frame in it i retrieved while the egment before it in the egment torage order i not, the number of frame in the retrieved egment that have to be dicarded equal p I + min(? p I? 1; q? 1). Here, p I denote the poition of the I frame within the egment. Proof: When a egment with an I frame in it i retrieved while the egment before it in the egment torage order i not, all frame belonging to the retrieved egment and that are ahead of the I frame (in the frame torage order) need to be dicarded. The number of uch frame equal p I. In addition, the (q? 1) B frame that follow the I frame (in the frame torage order) alo cannot be decoded and have to be dicarded. The number of uch B frame belonging to the retrieved egment equal min(? p I? 1; q? 1). 2 Property G7: When egment are retrieved conecutively (in the egment torage order) tarting with a egment containing an I frame, the maximum number of frame that have 10

11 to be dicarded equal p I + q? 1. Proof: A een in Property G6, when a egment with an I frame i retrieved while the egment ahead of it (in the egment torage order) i not, the p I frame belonging to the retrieved egment and ahead of the I frame (in the frame torage order) have to be dicarded. The (q? 1) B that follow the I frame (in the frame torage order) alo have to be dicarded. However, all frame that follow (in the frame torage order) the lat uch B frame that ha to be dicarded can be uccefully decoded. Hence, the reult. 2 Claim 1: When egment are retrieved conecutively (in the egment torage order) for the diplay of a piece of video, the rt uch retrieved egment hould be one with an I frame in it. Proof: An I frame i (either directly or indirectly) reponible for the ucceful decoding of the (qm? 1) frame that follow it in the frame torage order. A P frame i (either directly or indirectly) reponible for the ucceful decoding of the frame that follow it and that are before the next I frame (in the frame torage order). Hence, if a egment with no I frame in it i retrieved and the rt egment before thi (in the egment torage order) and containing an I=P frame i not retrieved, then all frame in thi retrieved egment have to be dicarded. 2 Out of the d egment et, we need to retrieve a certain number of egment o that the number of diplayed frame equal a. Following Claim 1, the retrieval of conecutive egment for the diplay of a piece begin with a egment containing an I frame. The maximum number of piece in d egment et equal the number of egment containing an I frame, which from Property G4 i d c qm. From Propertie G6 and G7, the number of retrieved frame that have to be dicarded for each piece varie between p I + min(?p I?1; q?1) and (p I +q?1). A tradeo exit between the number of piece and the number of retrieved frame that have to be dicarded. The more the number of piece, the more i the number of retrieved frame that have to be dicarded. In addition, (from Property G6) the earlier the poition of the I frame in the rt retrieved egment of a piece, the maller i the number of frame that have to be dicarded for that piece. For the particular grouping illutrated in Figure 4, Figure 5 illutrate the cae where n = 30 and n 1 = 54 fp. Out of b = 126 frame, we need to diplay a = 70 frame. Three (d = 3) egment et need to be conidered here and the maximum number of piece in thee b= = 63 egment i ix. The egment encloed in dahed boxe are thoe that contain I frame. One poible et of retrieved egment with three diplayed piece i alo hown in the gure. 2.4 Segment Retrieval Procedure S1 Once the number of piece (and conequently, the egment at the tart of the piece) i determined, the particular egment that are to be choen for retrieval may be determined 11

12 n=30, n1=54 a=70, b=126, d=3 Col c-1 Row 0 1 d b Retrieved Segment : Figure 5: Retrieval during fat playback uing the following procedure S1: Procedure S1 Input: b= egment with conecutive egment number k = number of piece q i = egment number at the tart of piece i = f0; 1; ; k? 1g a = number of frame to be diplayed out of b frame Output: Segment number of elected egment begin begin initialization a 1 = a (no. of frame belonging to currently unelected egment that need to be diplayed) i = 0 (piece to which currently choen egment belong) j = 0 (oet from tart of piece i) q = q i + j (egment number of currently choen egment for retrieval) end initialization while (a 1 > 0) do Select egment q Subtract no. of diplayed frame in egment q from a 1 12

13 end if (i < k? 1) do i = i + 1 ele do i = 0 j = j + 1 end if-ele q = q i + j end while Procedure S1 return the egment that need to be retrieved. The order in which thee egment are returned i not the ame a the retrieval order of egment. The retrieval order of egment i according to increaing egment number. 2.5 Advantage and diadvantage The primary advantage of the GGR grouping cheme are : The egment length can be choen to be any deirable value. Depending on the value of parameter uch a buer cot and available dik retrieval reource, a certain egment length may be preferred over other. The variance in egment ize i reaonably mall. Introduction of dummy frame at the tart and end of the video enure that no boundary eect exit during grouping. The formulae that are ued for grouping frame in the middle of the video are alo ued for grouping frame at the tart and end of the video. Cloed form equation provided facilitate grouping implicity. The primary advantage of the GGR retrieval cheme are : Any arbitrary playback rate (n 1 ) can be handled. The number of piece (and conequently the duration of each piece) can be varied depending on the deired appearance of the fat playback. The number of retrieved frame that have to be dicarded i mall. Procedure S1 provide a imple algorithm for determining which egment need to be retrieved. 13

14 The primary diadvantage of the GGR grouping cheme are : Dummy frame need to be introduced to circumvent boundary eect iue. Though the variance in egment ize i not very large, an even maller variance may be deirable. The primary diadvantage of the GGR retrieval cheme are : At leat ome frame belonging to retrieved egment have to be dicarded. The number of dicarded frame cannot be brought down to zero. The maximum number of piece i xed by the number of egment that contain an I frame. If qm i large and i mall, then the maximum number of piece i much maller than the total number of egment. An increae in the number of piece (relative to the total number of egment) may be deirable in ome cae, but cannot be realized. 3 DGR cheme The primary goal of the DGR grouping and retrieval cheme i to eliminate any dicarded frame during fat playback, i.e., all frame belonging to retrieved egment are diplayed. A een earlier, the frame torage order i a caual arrangement of frame uing which frame belonging to a video egment may be directly determined. Every et of conecutive frame beginning at the tart of the frame torage order are grouped together to form a egment. The minimal caual order, a we aw earlier, i one uch caual order. Obervation 1: When the minimal caual order i ued a the frame torage order, during fat playback ome frame belonging to retrieved egment may have to be dropped. Our primary aim here i to eliminate the dicarding of any retrieved frame. The frame torage order that achieve thi (a will be een later) i obtained by uitably modifying the minimal caual order (with the dummy B frame). Let denote the egment length. For every et of neighboring I=P frame tarting at the beginning of the minimal caual order (with the dummy B frame), the lat (? 1) I=P frame in thi et are moved ahead to immediately follow the rt I=P frame in thi et. The frame torage order depend on the egment length and i hown in Figure 6(d) for the cae = 4. The egment reulting from thi frame torage order are hown in Figure 7(a). 3.1 Grouping Propertie Each egment i aigned a egment number with the rt egment being aigned a number zero. We hall refer to the arrangement of egment in acending order of egment number 14

15 (a) Playback Order : I B B B B P B B B B P B B B B P B B B B P B B B B P B B B B Frame Number : q=5 qm=30 Minimal Caual Order : I P B B B B P B B B B P B B B B P B B B B P B B B B I B B B B (b) Poition ( i ) : Frame Num (i) : (c) Minimal Caual Order : I B B B B P B B B B P B B B B P B B B B P B B B B P B B B B (with dummy frame) Poition ( i ) : Frame Num (i) : 0 x x x x (d) Storage Order : I P P P B B B B B B B B B B B B B B B B (=4) P P I P B B B B B B B B B B B B B B B B P P P P B B B B B B B B B B B B B B B B Poition ( i ) : Frame Num (i): x x x x Figure 6: Frame torage order 15

16 a the egment torage order. Dene a egment et to be a et compriing the mallet number (ay, c) of conecutive egment in the egment torage order that form a egment pattern. The rt egment et include c conecutive egment beginning with the rt egment of the egment torage order, i.e., egment number 0. We will ue the following notation to denote dierent egment type : I 0 IP egment : A egment that contain an I frame in poition 0 and the remaining (? 1) frame are either I or P frame. P 0 IP egment : A egment that contain a P frame in poition 0 and the remaining (? 1) frame are either I or P frame with at leat one I frame. I egment : Either an I 0 IP or a P 0 IP egment, i.e., a egment with at leat one I frame. P egment : A egment that contain P frame. I=P egment : Either an I or a P egment. B egment : A egment that contain B frame. Property D1: The total number of egment in a egment et equal c = qlcm (m;). Proof: A egment i either an I=P egment or a B egment. Since every I=P egment in the egment torage order i followed by (q? 1) B egment, the number of egment in a egment et i q time the number of I=P egment in it. The number of I=P egment in a LCM (m;) egment et equal. Hence, the reult. 2 Property D2: (a) The egment with number iq for i = f0; 1; g contain frame with number iq; (i + 1)q; (i + 2)q; ; (i +? 1)q. (b) The egment with number (iq +j) for i = f0; 1; g and j = f1; 2; ; q?1g contain the l-th B frame (in the frame torage order), where l = i(q? 1) + (j? 1); i(q? 1) + (j? 1) + 1; i(q? 1) + (j? 1) + 2; ; i(q? 1) + (j? 1) +? 1. (c) The l-th B frame in the frame torage order ha a frame number bl=(q? 1)cq + (l mod (q? 1)) + 1? q. Here, l = f0; 1; g. Proof: (a) The egment with number iq for i = f0; 1; g i the i-th I=P egment in the egment torage order. The i-th I=P egment (in the egment torage order) contain the l-th I=P frame in the frame torage order, where l = i; (i + 1); (i + 2); ; (i +? 1). Thee frame have the number iq; (i + 1)q; (i + 2)q; ; (i +? 1)q. (b) The egment with number (iq + j) for i = f0; 1; g and j = f1; 2; ; q? 1g i the ((q? 1)i + j? 1)-th B egment in the egment torage order. The m-th B egment in the egment torage order contain the l-th B frame in the frame torage order where l = m; m + 1; m + 2; ; m + (? 1). Plugging in m = ((q? 1)i + j? 1), we have the reult. 16

17 (c) The l-th B frame in the frame torage order i alo the l-th B frame in the minimal caual order. The number of frame in the minimal caual order that are ahead of the l-th B frame and after being rounded down to the cloet multiple of q i bl=(q? 1)cq. Thi i the number of frame in the minimal caual order ahead of the l-th B frame a well a ahead of the I=P frame which i jut ahead of the l-th B frame. The oet of the l-th B frame from the I=P frame that i jut ahead of it (in the minimal caual order) i (l mod (q? 1)) + 1. Thu, the l-th B frame in the minimal caual order ha a poition of bl=(q?1)cq+(l mod (q?1))+1. A B frame in the playback order i q place ahead of it poition in the minimal caual order [10]. Hence, the reult. 2 Property D2 can be ued to determine the frame contained in any egment. Given any egment number, it provide cloed form equation to determine the frame number of each of the frame contained in the egment. 3.2 Fat Playback We hall now dicu egment retrieval during fat playback. The primary aim i to eliminate any dicarding of retrieved frame. Iue concerning piece length will alo be dicued. Let n and n 1 fp be the normal and fat playback rate repectively, where n 1 > n. On an average, n out of every n 1 frame need to be diplayed. Since all frame belonging to retrieved egment are diplayed, n out of every n 1 egment need to be retrieved. Canceling n the highet common factor (HCF) between the two, out of every n 1 egment HCF (n;n 1 ) HCF (n;n 1 ) need to be retrieved. Once the choice of egment to be retrieved i determined for an integral number of egment et, ubequent retrieval are determined in a trivial manner. Hence, we n need to conider the retrieval of LCM(c; 1 ) HCF (n;n n n 1 ) n 1 out of LCM(c; 1 ) egment. HCF (n;n 1 ) We hall denote the former value by a and the latter by b, each of which i an integer. Such a retrieval reult in the diplay of a f = a out of every b f = b frame. In the ret of the dicuion, we will aume that an I egment contain exactly one I frame, i.e., that m. The extenion to the cae where thi aumption i relaxed i conceptually imilar. Propertie for thi cae can be imilarly derived and we will not conider it here. Property D3: Out of the c egment in a egment et, the number of I egment equal LCM (m;) m. Proof: The number of I or P frame in a egment et equal LCM(m; ). Since each I egment contain exactly one I frame and there exit one I frame for every m conecutive LCM (m;) I or P frame, the number of I egment in a egment et equal. 2 m Property D4: The poition of the I frame in the i-th I egment of any egment et i mi mod. Here, i = f0; 1; ;? 1g. LCM (m;) m Proof: For every m conecutive I or P frame in a egment et, there exit one I frame. 17

18 Of all I or P frame in a egment et, the rt one i an I frame. An I=P egment contain conecutive I or P frame with the rt I egment in the egment et containing the I frame in poition 0. Hence, the reult. 2 Property D5: Out of the c egment in a egment et, the number of P egment between m?(?(mi mod )) LCM (m;) the i-th and the (i+1)-th I egment equal b c. Here, i = f0; 1; ;? m 1g. Proof: From Property D3, the poition of the I frame in the i-th I egment i mi mod. The number of P frame that follow the I frame in the i-th I egment i? (mi mod )? 1. The number of P frame (in the frame playback order) that lie between the lat P frame following the I frame in the i-th I egment and the next I frame equal m?(?(mi mod )). Since a egment contain frame, the number of P egment (if any) between the i-th and (i + 1)-th I egment equal b m?(?(mi mod )) c. 2 Since a egment et form a pattern of egment, Propertie D4 and D5 alo hold for the i-th I egment in the entire torage order (and not merely the egment et). Property D6: Out of the c egment in a egment et, the number of B egment equal c. q?1 q Proof: Every I=P egment in the egment torage order (and hence in the egment et) i followed by (q? 1) B egment. Since the number of egment in a egment et i c, we have the reult. 2 Property D7: The maximum number of P egment between any two neighboring I egment (in the egment torage order) i that between the lat I egment of a egment et and the rt I egment of the next egment et. Thi number equal b m?1 c. Proof: The rt I egment of a egment et i an I 0 IP egment, i.e., the I frame i in poition 0. The (m? 1) P frame ahead of thi I frame (in the frame torage or playback order) reult in b m?1 c P egment. If an I egment that i a P 0 IP egment i conidered, then the number of P egment ahead of thi egment i given by b m?1?p c, where p i the poition of the I frame in the P 0 IP egment. Since p > 0 for a P 0 IP egment, b m?1?p c never exceed b m?1 c. Hence, the reult. 2 For egment arranged according to the egment torage order, the following decoding fact hold : Decoding Fact 1. Any I 0 IP egment may be decoded independently. 2. Any P 0 IP or P egment only require the previou I=P egment for the ucceful decoding of all it frame. 18

19 3. The rt b q?1 c B egment following any I=P egment not only require thi I=P egment but alo the I=P egment prior to it for the ucceful decoding of all it frame. 4. The ubequent (q? 1? b q?1 c) B egment only require thi I=P egment for frame decoding. In the above, when a certain egment i required, we mean that the decoded frame of thi egment are required. Decoding the frame of the egment that i required may itelf require other egment. Baed on the above fact, in order to guarantee that no retrieved frame are dicarded, the following retrieval guideline need to be adhered to. (The guideline here have a one-to-one correpondence with the fact above.) : Retrieval Guideline 1. The I 0 IP egment need to be retrieved prior to the retrieval of any egment following it and preceding the next I 0 IP egment. 2. The I=P egment following an I 0 IP egment and preceding the next I 0 IP egment need to be retrieved according to increaing egment number, i.e., an I=P egment earlier in the torage order need to be retrieved prior to the retrieval of one appearing later in the order (for all I=P egment between two neighboring I 0 IP egment.) 3. Each of the rt b q?1 c B egment following an I=P egment can be retrieved after the retrieval of thi I=P egment a well a the I=P egment prior to it. 4. The ubequent (q? 1? b q?1 c) B egment can be retrieved after thi I=P egment ha been retrieved. In order to diplay a f out of b f frame, we need to retrieve a out of b egment. Sticking to the guideline above and baed on the deired ize of normal playback piece, everal egment retrieval algorithm are poible. Three poible retrieval are illutrated in Figure 7(b) for the cae n = 30, n 1 = 54 fp. Thi fat playback rate require a f = 100 out of b f = 180 frame to be diplayed, or a = 25 out of b = 45 egment to be retrieved. The egment encloed in the dahed box are the I=P egment while the other are B egment. In retrieval (iii), ince egment 40 i not retrieved, following guideline (3), the rt B egment following the next I=P egment (i.e., egment 46) cannot be decoded. For the ame reaon, egment 1 cannot be decoded and i not retrieved. 19

20 (a) Grouping into Segment Storage Order : IPPP BBBB BBBB BBBB BBBB PPIP BBBB BBBB BBBB BBBB PPPP BBBB BBBB BBBB BBBB q=5 q-1 =4 =1 Frame Number: 0,5,10,15 xxxx 1,2,3,4 6,7,8,9 11,12,13,14 20,25,30,35 16,17,18,19 21,22,23,24 26,27,28,29 31,32,33,34 40,45,50,55 36,37,38,39 41,42,43,44 46,47,48,49 51,52,53,54 Segment Number: c = q LCM(m,) =15 (b) Segment Skipping n=30, n1=54 a f = 100 b f = 180 a = 25 b = 45 d= 3 Col Row 0 1 d c b (i) Retrieved Segment (ii) (iii) Figure 7: (a) Segment grouping (b) Segment retrieval 20

21 3.3 Other Interactive Operation So far, we have been concerned with egment retrieval during fat playback in the forward direction. However, for application uch a VOD, the uer i permitted a variety of other interactive operation. We hall briey dicu the retrieval iue with the propoed grouping cheme during thee interactive operation. A low/normal forward play retrieve egment according to the egment torage order. All egment are retrieved, i.e., no egment are kipped. A jump forward/backward can only be made to an I frame belonging to an I 0 IP egment. If thi i not the cae (i.e., if the jump need to be made to a egment other than the I egment in an I 0 IP egment), ome frame belonging to retrieved egment will have to be dicarded. The decoding fact and retrieval guideline lited earlier are independent of the playback direction (forward or revere) or peed (low, normal or fat). When revere playback (of any peed) i conidered, the order in which egment are to be retrieved i dierent from that in the forward direction. We propoe Procedure R to determine the order of egment retrieval for low/normal revere playback, i.e., when all egment are retrieved. Procedure R Input: x 1 = egment et from which revere playback begin = number of I egment in a egment et LCM (m;) m Output: Segment in the order they hould be retrieved begin begin declaration i = egment et from which retrieval i currently taking place j = I egment to be retrieved from egment et i end declaration for i = x 1 to 0 do for j = LCM (m;) m? 1 to 0 do Retrieve j-th I egment in egment et i Retrieve P egment following thi I egment (and before the next one) in increaing egment number Retrieve B egment following the retrieved I=P egment in decreaing egment number end for 21

22 Time Retrieved egment Played back ,53 50,49 46,45 42,41 frame ,51 48,47 44,43 40,39 # egment in buer ,37 34,33 30,29 26,25 22,21 18,17 14,13 10,9 6,5 36,35 32,31 28,27 24,23 20,19 16,15 12,11 8,7 4, Table 3: Segment retrieval for low/normal revere playback end end for Theorem 1: For low/normal revere playback uing the propoed grouping cheme, Procedure R reult in the leat number of prefetched egment prior to tart of playback (in order to avoid video tarvation). Thi number equal 1 + b m?1 c. Proof: From Property D7, we know that the maximum number of P egment between neighboring I egment i b m?1 c and that thi occur at the end of the egment et. For the decoding of the B egment that follow the lat P egment in the egment et, the P egment itelf i needed. For the decoding of thi P egment, the P egment prior to it i needed, and o on until the I egment i reached. Thu, to decode the B egment that follow the lat P egment of a egment et, we require all I=P egment from the lat I egment in the egment et. Thi number equal 1 + b m?1 c, with the 1 being for the I egment. Hence, at leat 1+b m?1 c egment have to prefetched before the lat B egment i retrieved and played back. Thi alo avoid video tarvation becaue no other egment require more egment for the ucceful decoding of all it frame. 2 For the egment in Figure 7, the order of egment retrieval uing Procedure R and the revere frame diplay i illutrated in Table 3. A in the cae of fat forward, fat revere need to kip retrieval of certain egment. The choice of egment to kip again depend on the ame two iue that were conidered in the fat forward cae - deired eect of fat diplay (i.e., ize of piece) and the et of retrieval guideline. 22

23 3.4 Advantage and diadvantage The advantage of the DGR grouping cheme are : The egment length () can take on any deired value. While propertie 3, 4 and 5 aume m, imilar propertie may be obtained for the cae > m. All other propertie and the retrieval guideline hold good irrepective of the value of. Determination of which frame are grouped together into a egment can be eaily done uing the cloed form equation provided. The advantage of the DGR retrieval cheme are : No frame belonging to retrieved egment are dicarded. Compared to other cheme that dicard ome retrieved frame, the amount of retrieved video and hence the load on the dik retrieval reource i le. Thi can facilitate the upport of a larger number of uer for the ame dik retrieval reource, and hence reult in a more cot eective ytem. Several imple retrieval algorithm may be eaily devied baed on the deired fat playback appearance. Depending on the deired duration (and number) of normal playback piece, a uitable retrieval algorithm may be devied for fat playback. The diadvantage of the DGR grouping cheme are : The variance in egment ize i large. Thi i becaue an I=P egment contain only I=P frame while a B egment contain only B frame. The ize of an I=P frame i much larger than that of a B frame. A large variance in egment ize ha at leat two drawback: { Segment are uually triped acro a certain number of dik. The fragment of a egment that i tored in a dik i tored in a certain number of dik block. With a contant egment ize, the number of dik to tripe a egment acro can be o choen that a egment fragment (almot) completely occupie an integral number of dik block. With a large variance in egment ize, only a mall portion of a dik block may be ued to tore the lat portion of the fragment of a egment. Since the entire dik block ha to be retrieved to retrieve thi mall portion of data, dik retrieval reource are being wated. { Retrieval cheme for VOD application uually take place in ervice round. A large variance in egment ize can lead to large variation in the retrieval duration of egment and conequently quality of ervice guarantee are more dicult to make. 23

24 4 Concluion In thi paper, we propoed two cheme - GGR and DGR - for grouping MPEG frame into egment of any deired length. Cloed form equation were provided that can facilitate the determination of which frame need to be grouped together to form a egment. Guideline were provided for handling a variety of interactive operation - low/normal/fat forward/revere play and jump forward/backward. Uing thee guideline and depending on the deired eect of fat playback, uitable retrieval algorithm may be eaily devied. Due to the propertie enjoyed by the minimal caual order, it wa choen to be the frame torage order in GGR. During fat forward/revere playback, retrieval of ome egment need to be kipped. The choice of egment to retrieve depend on the deired number of video piece a well a on the maximum number of retrieved frame that can be dicarded. Uing the propertie of GGR provided, a uitable retrieval cheme can be devied. One uch retrieval algorithm (Procedure S1) wa provided. The DGR cheme overcome the drawback of GGR that ome frame belonging to retrieved egment have to be dicarded during fat playback. Such cheme can be ued for a variety of emerging application, uch a VOD, which require tored MPEG video. Reference [1] M.S. Chen, D.D. Kandlur, and P.S. Yu. \Storage and Retrieval Method to upport fully interactive playout in a dik-array-baed video erver". Multimedia Sytem, 3:126{135, [2] D. Deloddere, W. Verbiet, and H. Verhille. \Interactive Video On Demand". IEEE Comm. Mag., page 82{88, May [3] Y.N. Doganata and A.N. Tantawi. \Making a cot-eective Video Server". IEEE Multimedia, [4] T.L. Kunii, Y. Shinagawa, R.M. Paul, M.F. Khan, and A.A. Khokar. \Iue in Storage and Retrieval of Multimedia Data". Multimedia Sytem, 3:298{304, [5] V.O.K. Li and W.J. Liao. \Ditributed Multimedia Sytem". Proc. of the IEEE. Accepted. [6] V.O.K. Li, W.J. Liao, X. Qiu, and E.W.M. Wong. \Performance model of interactive video-on-demand ytem". IEEE JSAC, Augut [7] Pramod Pancha and Magda El Zarki. \A look at the MPEG video coding tandard for variable bit rate video tranmiion". IEEE INFOCOM, page 85{94,

25 [8] Pramod Pancha and Magda El Zarki. \MPEG Coding for Variable Bit Rate Video Tranmiion". IEEE Communication Magazine, page 54{66, May [9] K. Salem and H. Garcia-Molina. \Dik Striping". Proc. Second International Conf. on Data Engg., page 336{342, [10] S. Sengodan and V.O.K. Li. \A Generalized Grouping and Retrieval Scheme for Stored MPEG Video". IEEE ICC, June [11] S. Sengodan and V.O.K. Li. \A Quai-tatic Retrieval Scheme for Interactive VOD Server". Computer Communication, May [12] S. Sengodan and V.O.K. Li. \A Shared Buer Architecture for Interactive VOD Server". IEEE INFOCOM, April