TRADING DCT/IDCT QUALITY FOR ENERGY REDUCTION IN MPEG-2 VIDEO CODECS

Transcription

1 PLEASE NOTE THAT THIS SUBMISSION IS INTENDED SPECIFICALLY FOR THE SPECIAL ISSUE ON LOW POWER ELECTRONICS AND DESIGN TRADING DCT/IDCT QUALITY FOR ENERGY REDUCTION IN MPEG-2 VIDEO CODECS Russell Henning and Chaitali Chakrabarti Department of Electrical Engineering Arizona State University Tempe, AZ Ph: (480) Fax: (480) Abstract It would be desirable, in terms of energy conservation, to use one low complexity, approximate algorithm to do all discrete cosine transform (DCT) and inverse discrete cosine transform (IDCT) computation in an MPEG-2 video codec. However, there is a significant quality penalty associated with this approach that may not always be acceptable. A technique that applies both exact and approximate DCT/IDCT algorithms is presented for achieving a more practical tradeoff between quality and energy in MPEG-2 video codecs. Additionally, multiple quality/energy tradeoff modes can be supported with this technique to allow energy reduction scalability. This functionality is studied for a number of communication system configurations. For example, one of these systems consists of two video codecs communicating with one another. With the new technique, the energy consumption of DCT and IDCT execution in the two video codecs can be reduced to different levels depending on the dynamic needs of the application. In a conventional experimental setup employing the Flower Garden test sequence, this energy is reduced by an estimated 8% for video quality reduction of 0.4 db average PSNR, 14% for 0.8 db reduction, or 22% for 1.4 db reduction. This work was carried out at the National Science Foundation's State/Industry/University Cooperative Research Centers' (NSF-S/IUCRC) Center for Low Power Electronics (CLPE). CLPE is supported by the NSF (Grant #EEC ), the State of Arizona, and the following companies and foundations: Burr- Brown, Inc., Conexant, Gain Technology, Intel Corporation, Medtronic Microelectronics Center, Microchip Technology, Motorola, Inc., The Motorola Foundation, Raytheon, Texas Instruments and Western Design Center.

2 1 INTRODUCTION Over the past decade, increasing demand for portability has made design of low power VLSI systems increasingly important. In early development of a VLSI implementation, power reduction opportunities abound. Power can be reduced at all levels (algorithm, architecture, circuit, and process levels) of such designs [1, 2]. However, in a typical top-down design, opportunities at each level are limited by constraints set at higher levels. An important example of design freedom that is generally lost if not taken advantage of during high-level (algorithm and architecture level) design is the opportunity to reduce energy through exploitation of data characteristics. In this paper, a technique is presented that uses the often overlooked potential of data-dependent low power design to reduce significant power consumption in MPEG-2 video codecs at the expense of a reasonable amount of video quality. Though this particular approach is developed for a specific application, it is a valuable example for understanding how the potential of data-dependent low power design can be tapped in other applications as well. It would be desirable, in terms of energy conservation, to use one low complexity, approximate algorithm to do all discrete cosine transform (DCT) and inverse discrete cosine transform (IDCT) computation in an MPEG-2 video codec. These modules are among the greatest energy consumers in encoder and decoder implementations, and low power versions could certainly be advantageous to future wireless applications. However, there is a significant quality penalty associated with this approach that may not always be acceptable. Therefore, a new approach is proposed that reduces energy at the expense of a more acceptable level of video quality by exploiting differences between the data that is processed for I, P, and B-type frames during DCT/IDCT computation. Both exact and approximate DCT/IDCT algorithms are applied according to data characteristics of each frame type in order to achieve the improved quality/energy tradeoff. For 2

3 example, unlike I and P frames, B frames are never referenced when inter frame coding is applied to macroblocks in P and B frames. Therefore, errors introduced into B frames through approximate IDCT processing at the decoder cannot be propagated to other nearby frames the way they can for I and P frames. This characteristic of B frames can be exploited with the new approach to reduce energy at the expense of less quality loss by choosing to use approximate IDCTs for B frames as opposed to I or P frames. The new data-dependent execution approach can also support energy reduction scalability (similar in concept to bitrate scalability). In certain wireless MPEG video applications, energy reduction and video quality needs can potentially vary with time. This variation can be accommodated if encoders and/or decoders allow dynamic scaling of the quality/energy tradeoff through use of the new approach. Take, for example, a communication system consisting of two such video codecs communicating with one another. With the new technique, the energy consumption of DCT and IDCT execution in the two video codecs can be reduced to different levels depending on the dynamic needs of the application. In a conventional experimental setup employing the Flower Garden test sequence, this energy is reduced by an estimated 8% for video quality reduction of 0.4 db average PSNR, 14% for 0.8 db reduction, or 22% for 1.4 db reduction. Six communication configurations in all have been analyzed experimentally to determine their energy reduction scalability potential. The experiments employed a single approximate DCT/IDCT algorithm, which was developed as part of this investigation. Other approximations could certainly be used but would lead to different quality/energy tradeoff levels. The configurations considered are as follows: 1) One-way communication between a conventional encoder and energy sensitive decoder 2) One-way communication between an energy insensitive encoder and energy sensitive decoder 3

4 3) One-way communication between an energy sensitive encoder and energy insensitive decoder 4) One-way communication between an energy sensitive encoder and energy sensitive decoder 5) Two-way communication between an energy sensitive terminal and energy insensitive terminal 6) Two-way communication between two energy sensitive terminals In Section 2, related work in the area of data-dependent power reduction techniques will be reviewed. Section 3 presents the new quality/energy tradeoff approach for MPEG-2 codecs in detail and explains how energy reduction scalability is achieved. Section 4 compares estimated energy consumption of different quality/energy tradeoff options. Section 5 gives experimental results and analysis of energy reduction scalability potential for the six communication configurations tested. Section 6 summarizes key findings and outlines areas for future study. 2 RELATED WORK The potential of high-level design was emphasized in early work on power reduction for digital circuits [2, 3]. In spite of this, it is one of the least explored areas in low power design. Until recently, most work in this area focussed on filter implementations. Since the greatest potentials for power reduction at this level tend to be application dependent, more data-dependent design approaches and examples in a variety of application domains are needed. The existing work in algorithm level power reduction has been geared primarily towards control/data flow graph optimizations [2], with a few exceptions [1, 4]. Algorithm level power reduction can be obtained by control/data flow graph optimizations such as control step reduction, operation reduction, operation substitution, resource reduction, transition activity reduction, and word length reduction [2]. 4

5 Most of the algorithm design work in which data characteristics have been exploited is in the area of filter design. The data characteristics of the filter coefficients have been exploited and manipulated to achieve significant power reduction. Static techniques include use of powers-oftwo coefficients [5], use of differential filter coefficients [6], reduction of Hamming distance between consecutive coefficients, coefficient ordering, and selective coefficient negation [7]. More impressive gains have been achieved in filters when signal nonstationarity is exploited. In these examples, the number of filter taps and coefficient values are varied based on a filter quality factor such as output stopband energy attenuation, MSE between the output and the desired signal, or SNR/BER of the output [8-10]. Reduction in the number of filter taps causes a reduction in switching as well as a reduction in the length of the critical path (which can be traded off for reduction in supply voltage). The overhead of this approach is the power consumption of the signal monitoring algorithm. Use of this technique in a speech signal subband decomposition example was shown to reduce power by 10 times with minimal quality loss [8]. In a near-end cross talk canceller, an average of 62% power reduction was achieved [10]. This average increased to 69% when supply voltage reduction was also considered. In a very high speed digital subscriber loop equalizer, 88% average reduction was achieved by adjusting the filter taps and reducing the supply voltage [9]. Algorithmic power reduction techniques have been applied to only a few examples that are not filter applications. These applications include vector quantization [1], vector quantization decoding [4], and, most recently, encryption [11] and channel coding [12]. The more recent examples consider dynamic variations in data. For example, Goel and Shanbhag showed that the energy consumption of a particular Reed-Solomon codec implementation can be reduced by 55% on average by powering down taps that are not required to meet a desired bit error rate for an input with dynamically varying SNR. By translating variations in the workload to variations in the supply voltage, significant power savings have been obtained in DSP processors as well [13]. 5

6 3 ENERGY REDUCTION APPROACH A new data-dependent approach to trading off quality for energy consumption in MPEG-2 video codecs will now be described. It begins with selection of low energy, exact and approximate DCT/IDCT algorithms. The approach involves mixing execution of these algorithms according to frame type to achieve a desired quality/energy tradeoff. Energy reduction scalability is achieved by supporting multiple configurations that tradeoff quality for energy differently. 3.1 Exact and Approximate DCT/IDCT Algorithms One exact and one approximate 8 point 1-D DCT/IDCT algorithm have been chosen to study the effectiveness of this approach. These 1-D DCT/IDCT algorithms are applied in an MPEG-2 video codec using the Row-Column 2-D DCT/IDCT. The exact DCT/IDCT algorithm chosen requires only 5 multiplications and 29 additions [14]. An explicit derivation of this algorithm can be found on the web [15]. We call this method the Scaled Exact (SE) 1-D DCT/IDCT algorithm, because outputs of the DCT algorithm and inputs to the IDCT algorithm are scaled. The algorithm actually requires 13 multiplications per 1-D DCT/IDCT, however the 8 scaling multiplications can be combined with multiplications in the quantization or inverse quantization calculations of the MPEG-2 codec through pre-computation. An approximate 1-D DCT/IDCT algorithm can be employed to achieve fewer than 5 multiplications and thus conserve more energy. The approximation chosen for this study was first introduced in [16]. It is an 8 point 1-D DCT/IDCT approximation that uses no multiplications and only 28 additions. We refer to this algorithm as the Scaled Approximate (SA) 1-D DCT/IDCT because outputs of the DCT algorithm and inputs to the IDCT algorithm are scaled in the same manner as the SE algorithm. The SA algorithm is an approximation of the exact 1-D DCT/IDCT algorithm in [17]. Similar approximations can be found in [18]. The 8 point 1-D DCT can be written in matrix form as 6

7 D * x = X, where x and X are column vectors of dimension 8X1 and D is a matrix of dimension 8X8. By transposing both sides of this equation, we get the form that is typically implemented, x t * D t = X t. Vetterli and Ligtenberg [17] proposed a fast algorithm based on a decomposition of the D matrix, which can be expressed as follows if some of the stages are combined D t = S1 * S2 * S3 * S4 * W where S1 = S2 = C5 -C S3 = 0 0 C6 C C10 C7 -C8 -C C7 C9 C10 -C C8 C10 -C9 -C C9 C8 C7 C S4 =

8 C W = 1/2 * C C4 = 2 π C5 = sin( 8 ) π C6 = cos( 8 ) π C7 = sin( 16 ) π C8 = cos( 16 ) 3π C9 = sin( 16 ) 3π C10 = cos( 16 ). The SA 1-D DCT algorithm is obtained by replacing the coefficients in the above decomposition with the following approximate values to allow multiplications to be computed with simple shifts: C5 = 0.5 C6 = 1 C7 = 0.25 C8 = 1 C9 = 0.5 C10 = 1 The exact IDCT can be expressed similarly as where X t * D = x t D = W * S4 t * S3 t * S2 t * S1 t and the exact coefficients are used. The SA 1-D IDCT algorithm is found by applying the same coefficient approximations as were used to obtain the SA 1-D DCT algorithm. Pseudo-code for 8

9 /* coefficients */ /* first stage */ /* second stage */ /* third stage */ C5 = 0.5; a0 = X0; tmp1 = a0+a1; x0 = b0+b7; C7 = 0.25; a1 = X1; tmp2 = a0-a1; x1 = b1+b6; C9 = 0.5; a2 = C5*X2-X3; b0 = tmp1+a3; x2 = b2+b5; C10 = 1; a3 = X2+C5*X3; b1 = tmp2+a2; x3 = b3+b4; a4 = C10*X4+C7*X5-X6-C9*X7; b2 = tmp2-a2; x4 = b3-b4; a5 = C7*X4+C9*X5+C10*X6-X7; b3 = tmp1-a3; x5 = b2-b5; a6 = -X4+C10*X5-C9*X6-C7*X7; b4 = a4; x6 = b1-b6; a7 = C9*X4+X5+C7*X6+C10*X7; b5 = a5; x7 = b0-b7; b6 = a6; b7 = a7; Figure 3.1. Pseudo-code for the Scaled Approximate 1-D IDCT algorithm the SA 1-D IDCT algorithm is shown in Figure 3.1. The X's in this code are IDCT inputs, and the x's are IDCT outputs. Multiplications in the pseudo-code can be implemented with simple binary shifts or no operation at all because of the coefficient approximations. 3.2 Algorithm Mixing Methods The quality/energy tradeoff approach for MPEG-2 video codecs boils down to choosing, in a data-dependent manner, which frames should employ the approximate DCT/IDCT and which frames should use the exact DCT/IDCT. Different methods of mixing execution of the SE and SA algorithms will achieve different quality/energy tradeoffs. Therefore, the appropriate algorithm mixing method must be identified to achieve the desired tradeoff. Two methods of mixing execution of the SE and SA algorithms are investigated here for MPEG-2 video encoder DCTs and decoder IDCTs. The first is to use the SE algorithm for I and P frames of a group of pictures (GOP) and the SA algorithm for B frames. In a typical GOP, like one of the form IBBPBBPBBPBBPBB, this method allows the lower energy SA algorithm to be used for 2/3 of the frames in a sequence, while the SE algorithm is used for only 1/3. This algorithm mixing method is based on two characteristics of a B frame. First, unlike I and P frames, B frames are not referenced when inter frame coding is applied to macroblocks in P and B frames. Thus, when this method is applied to decoder IDCTs, quality reduction associated with approximate processing is confined to B frames, rather than spread to other frames in the GOP. When this method is applied to encoder DCTs, I and P frame quality is not affected 9

10 directly by B frame approximation. However, I and P frame quality can be affected by rate control, because B frame encoding efficiency is lost. The second characteristic of a B frame is that more macroblocks of B frames tend to be inter frame coded than P frames. Only the residual portion of such a macroblock is distorted by applying the SA DCT in the encoder or SA IDCT in the decoder to the macroblock. Therefore, these macroblocks tend to be more robust to associated quality degradation than intra coded macroblocks. As a result, distortion is potentially reduced for frames with more inter frame coded macroblocks. The second algorithm mixing method considered here that can be applied to either encoder DCTs or decoder IDCTs uses the SE algorithm only for the I frames, while the SA algorithm is used for P and B frames. With this method, the lower energy SA algorithm is used for 14/15 of the frames in the example GOP, and the SE algorithm is used for only 1/15. This method is based on characteristics of I and P frames. Since an I frame can be the basis for inter frame coding throughout a GOP, its integrity is in the greatest need of preservation. Additionally, unlike an I frame, a significant number of macroblocks of a P frame tend to be inter frame coded, though less than in a B frame. As was the case for B frames, these inter frame coded macroblocks tend to be more robust to approximation than intra coded macroblocks. As for the encoder IDCT, only one algorithm mixing method is considered here: IDCTs applied to I frames are computed with the SE algorithm, while IDCTs applied to P frames are computed with the SA algorithm. (Only I and P frames are processed with encoder IDCTs.) With this method, the SA algorithm is used for 4/5 of I and P frames in the example GOP, while the SE algorithm is used for 1/5. By approximating the encoder IDCT, coding efficiency of macroblocks that use the affected reference frame in the encoder will be decreased. Quality may be further impacted if the same approximation is not used for the same frame in the decoder, because encoder and decoder reference frames for inter frame coding will not match. 10

11 Other algorithm mixing methods, like only approximating subsets of P and/or B frames in a GOP are also possible, but the most important aspects of the new quality/energy tradeoff approach can be studied with the algorithm mixing methods that have just been described. 3.3 Energy Reduction Scalability In certain wireless MPEG video applications, energy reduction and video quality needs can potentially vary with time. The level of energy reduction needed and quality degradation that is acceptable can depend on variables such as the user, the video content, and the state of the power supply. This variation can be accommodated if encoders and/or decoders allow dynamic scaling of the quality/energy tradeoff through use of the new approach. This energy reduction scalability can be achieved by supporting multiple quality/energy tradeoff options and having a way to choose which of these options should be used at a particular time. The best configurations for achieving energy reduction scalability with the algorithm mixing methods presented in the previous subsection will be analyzed for six practical communication systems in Section 5. Key experimental results that led to selection of some of the configurations, like the importance of matching encoder IDCT and decoder IDCT methods and preserving I frame quality, will be discussed. 4 ENERGY COMPARISONS Differences in the number and type of operations performed by each of the DCT/IDCT computation methods described in the previous section can lead to significant energy consumption in a VLSI implementation. To compare the different methods, we consider highlevel energy consumption estimates for multipliers, adders, transposition memory, and control logic. Binary shifts used to avoid overflows and implement shift-and-add multiplications are assumed to be hard-wired in these estimates, so they have little effect on overall energy consumption. The SE and SA algorithms were found to produce quality near that of double 11

12 Relative Energy SE(IPB) SE(IP)SA(B) SE(I)SA(PB) SA(IPB) Method Figure 4.1. Relative energy estimates of mixing methods for either the encoder DCT or decoder IDCT Relative Energy SE(IP) SE(I)SA(P) SA(IP) Method Figure 4.2. Relative energy estimates of mixing methods for the encoder IDCT precision versions of each algorithm with 16 bit integer multipliers, 16 bit integer adders, and a 64 word SRAM using 16 bit words. Relative energy per operation estimates for multiplication, addition, and memory accesses can be found by assigning a certain functional unit (a 16 bit addition in this case) a normalized energy per operation value of 1. The other operations are then assigned energy per operation values relative to the 16 bit addition. Here, we choose to assign a relative energy value of to a 16 bit multiplication, to an SRAM read, and to an SRAM write. These values are based on general relationships published in [19] but reflect scaling of energy with respect to memory size. To obtain relative energy estimates for the DCT/IDCT methods, a 15 frame GOP of the form IBBPBBPBBPBBPBB was assumed. The estimates have been normalized by the energy of the case where the SE algorithm is used for all encoder DCTs or decoder IDCTs. This case consumes the most energy of all methods considered. The control energy estimate used is 20% of 12

13 this highest energy case and remains constant for all methods, except for the encoder IDCT where it is 1/3 of this value. Figure 4.1 shows the relative energy consumption of either the encoder DCT or decoder IDCT, since both consume the same amount of power for the same method. Figure 4.2 shows the relative energy consumption of the encoder IDCT. The notation used in these figures indicates which frame type uses which algorithm. For example, SE(IP)SA(B) indicates that I and P frame DCTs (or IDCTs) are computed with the SE algorithm, while B frames employ the SA algorithm for DCT (or IDCT) computation. Note that encoder IDCTs apply only to I and P frames, since B frames are not referenced in inter frame coding. 5 EXPERIMENTAL RESULTS AND ANALYSIS To distinguish the DCT/IDCT computation methods that have been considered thus far in terms of the tradeoff between quality and reduced energy consumption, the needs of specific applications should be considered. These tradeoff methods can be employed in one-way and twoway video communication where the energy reduction and video quality needs of the communicating devices may be different. By supporting multiple tradeoff methods, energy reduction scalability can be enabled in these applications. To demonstrate the differences between the tradeoff methods that best suit the needs of different applications, experimental results will be given and analyzed for six of the most practical communication system cases. Experiments were run using the MSSG MPEG-2 video codec. Four standard video sequences were considered, Flower Garden, Football, Mobile, and Table Tennis. Each sequence had a frame resolution of 352X240 pixels, color subsampling of 4:2:0, and frame rate of 30 frames per second. This resolution was chosen because it is expected that the primary applications of this technique would be in hand-held devices capable of relatively good quality. Each sequence was encoded with a 15 frame GOP structure of the form IBBPBBPBBPBBPBB. Fixed encoding bitrates of 4, 3, 2, and 1 Mbps were initially tested using the rate control technique included in the 13

14 MSSG codec. However, as will be discussed momentarily, a fixed bitrate of 4 Mbps is adequate for demonstrating the capabilities of these techniques. Quantitative quality results were obtained in terms of PSNR measurements. The authors made subjective visual observations of quality and verified that they closely follow the quantitative results. A nice feature of approximating DCT/IDCTs according to frame type is that the degradation is generally spread over the entire frame, rather than concentrated in a few macroblocks. Thus, the perceptual quality loss associated with steps of 0.5 db average PSNR loss is usually quite acceptable. As frames degrade, the quality loss becomes most apparent in dark, smooth-textured parts of the video where lighter noisy texture replaces the original smooth texture. Given the methods in Figures 4.1 and 4.2, there are 48 different ways in which the encoder DCT, encoder IDCT, and decoder IDCT can be configured for one-way video communication. However, for a particular application, a relatively small subset can be found to provide a useful variety of quality/energy tradeoff options. This subset can be used to support energy reduction scalability. The best subset of configurations identified through experimentation for six different communication cases will be discussed next. The cases are as follows: 1) One-way communication between a conventional encoder and energy sensitive decoder 2) One-way communication between an energy insensitive encoder and energy sensitive decoder 3) One-way communication between an energy sensitive encoder and energy insensitive decoder 4) One-way communication between an energy sensitive encoder and energy sensitive decoder 5) Two-way communication between an energy sensitive terminal and energy insensitive terminal 6) Two-way communication between two energy sensitive terminals 14

15 Average PSNR (db) SE(IPB) SE(IP)SA(B) S E (I)S A(P B ) SA(IPB) Rate (Mbps) Figure 5.1. Rate-distortion curves for the Flower Garden sequence conventionally encoded with a 15 frame GOP of form IBBPBBPBBPBBPBB and decoded with various methods An example of the notation that will be used to describe a particular system configuration for oneway communication is SE(I)SA(PB)/SE(IP)/SE(IP)SA(B). This notation means that SE(I)SA(PB) is used for the encoder DCT, SE(IP) is used for the encoder IDCT, and SE(IP)SA(B) is used for the decoder IDCT in the communication path. 5.1 Conventional Encoder, Energy Sensitive Decoder This first case involves one-way communication between a conventional encoder and energy sensitive decoder. An example of such a system is a wireless handset that can receive streaming video from a tethered server in order to display movies, etc. but cannot itself transmit video. Unlike the decoder in this case, the conventional encoder does not have the option of using quality/energy tradeoff methods. Figure 5.1 shows resulting rate-distortion curves for the Flower Garden sequence in this case. The other three video sequences tested exhibited similar results. Consider first the 4 Mbps average PSNR values. The quality results obtained by applying the SE(IPB) and SE(IP)SA(B) methods at the decoder IDCT are well within 0.5 db average PSNR of 15

16 Enc DCT Enc IDCT Dec IDCT Avg PSNR Reduction Decoder IDCT Energy % Reduction SE(IPB) SE(IP) SE(IPB) % SE(IPB) SE(IP) SE(IP)SA(B) % SE(IPB) SE(IP) SE(I)SA(PB) % Table 5.1. Quality/energy tradeoff of best configurations for a one-way communication system involving an energy sensitive decoder and a conventional encoder each other. The quality resulting from application of the SE(I)SA(PB) method is significantly less at 1.64 db below the SE(IPB) result. The SA(IPB) value drops even more to 2.70 db less than the SE(IPB) result. Taking these quality results and decoder IDCT energy estimates into consideration for this case, a good choice for implementing an energy reduction scalable MPEG-2 decoder for the 4 Mbps bitrate would be to enable decoding with the SE(IPB), SE(IP)SA(B), and SE(I)SA(PB) methods. These superior tradeoff options are summarized in Table 5.1 along with various tradeoff statistics. The average PSNR is included for the luminance component of the output when the Flower Garden sequence is tested. Next, the associated reduction in average PSNR with respect to the SE(IPB)/SE(IP)/SE(IPB) configuration is shown. The relative energy consumption estimates for the decoder IDCTs are included, and the associated percentage reduction in decoder IDCT energy with respect to the SE(IPB)/SE(IP)/SE(IPB) configuration is shown. An implementation that supports each of the tradeoff options in this table would allow very high quality decoding when executing the SE(IPB) method. Nearly as high quality decoding is achieved with the SE(IP)SA(B) method but at significantly lower IDCT execution energy, about 18.4% less than the SE(IPB) algorithm. This is actually quite a remarkable result, because by mixing IDCT algorithms in this manner, the SA algorithm can significantly reduce IDCT energy dissipation while giving up nearly insignificant quality. Using SA(IPB) gives considerably poorer quality than SE(IPB), about 2.70 db worse for the Flower Garden sequence, while reducing energy by an additional 9.3% compared to SE(IP)SA(B). Similar energy reduction (7.4% rather than 9.3%) can be achieved by mixing 16

17 SE(IPB) SE(IP)SA(B) SA(IPB) 35 PSNR (db) Frame Figure 5.2. SE(IP)SA(B) quality comparison for the first 46 frames of the Flower Garden sequence decoded from the 4 Mbps bitstream algorithms according to SE(I)SA(PB), but with only a 1.64 db decrease in quality for the Flower Garden sequence. Thus, SE(I)SA(PB) is probably a better alternative than SA(IPB) in terms of the tradeoff between quality and energy. As can be seen from Figure 5.1, all curves converge as the bitrate decreases. Note, however, that energy dissipation for each algorithm mixing method remains the same regardless of bitrate. This convergence hints at the importance of selecting an approximate DCT/IDCT algorithm that adequately fits the given bitrate, so that it can provide a useful variety of quality/energy tradeoffs as in the 4 Mbps case. Since the SA algorithm provides adequate examples for the purposes of this paper and the lower bitrates perform similarly with less difference between tradeoff options, 4 Mbps cases will be analyzed from this point forward. For more detail about how the algorithms of Figure 5.1 achieve their average quality, PSNR results for consecutive frames of the sequence can be studied. PSNR results for the first 46 frames of the Flower Garden sequence decoded from the 4 Mbps bitstream are shown in Figure 5.2 for SE(IPB), SE(IP)SA(B), and SA(IPB). Lower bitrate decoding exhibits similar curves with 17

18 SE(IPB) SE(I)SA(PB) SA(IPB) 35 PSNR (db) Frame Figure 5.3. SE(I)SA(PB) quality comparison for the first 46 frames of the Flower Garden sequence decoded from the 4 Mbps bitstream proportionally less significant PSNR differences between the algorithms. In this plot, it is possible to see that not only are the I and P frames significantly better in SE(IP)SA(B) compared to SA(IPB), the quality of the B frames are remarkably improved to near that of the SE(IPB) case. This is because the I and P frames, which are decoded with the exact IDCT, are used as differential coding references for inter frame coded macroblocks in B frames. A similar plot is shown in Figure 5.3 that compares the SE(I)SA(PB) method to SA(IPB) and SE(IPB). The SE(I)SA(PB) method tends to converge toward the quality of SA(IPB) over the 15 frame GOP until it is increased once again for the last two B frames of the GOP. This increase in quality occurs because the next I frame, which is decoded with the higher quality SE algorithm, is used as a reference for some differentially coded macroblocks. The SE(I)SA(PB) method has the side benefit, as does SA(IPB), of reducing the quality difference between P frames and B frames compared to SE(IPB) to provide a more uniform quality video sequence from frame to frame of a fixed rate bitstream. 18

19 One might intuitively expect using an exact algorithm for some frames and an approximation for others to cause a noticeable variance in frame quality. However, this variance is less than one might expect when using fixed bitrate bitstreams with the SA algorithm. In the case of SE(IP)SA(B) (Figure 5.2), low variance is a combination of two things. First, the approximation is fairly high in quality. Second, overall B frame quality is raised near that of the SE algorithm because many macroblocks where the SA IDCT is applied are encoded differentially using exact coded macroblocks as references. The difference between B frames using SE and SA IDCTs becomes even smaller for lower bitrates. In the case of SE(I)SA(PB) (Figure 5.3), the quality of P and B frames is significantly lowered. However quality variance between frames is also lowered. For fixed rate encoding, P frames tend to have much higher quality than other frames when all frames are decoded using exact IDCTs. Since P frames are decoded with approximate IDCTs in this case, the quality of P frames is lowered, so quality is more uniform from frame to frame. 5.2 Energy Insensitive Encoder, Energy Sensitive Decoder The next one-way communication case involves communication between an energy insensitive encoder and energy sensitive decoder. An example of such a system is once again a wireless handset that can receive streaming video from a tethered server in order to display movies, etc. but cannot itself transmit video. However, in this case, the encoder has the option of using one of the algorithm mixing methods. This capability can significantly improve some quality/energy tradeoffs for the energy sensitive decoder compared to the previous case where a conventional encoder was always used. Figure 5.4 shows quality results obtained with the Flower Garden sequence for possible oneway communication system configurations. From the four plots in Figure 5.4, it is clear that given one of the four mixing methods is applied at the decoder, one of the three encoder IDCT mixing methods produces the best result. The corresponding encoder IDCT that produces the best 19

20 Decoder IDCT Method = SE(IPB) Decoder IDCT Method = SE(IP)SA(B) Average PSNR (db) SE(IPB) SE(IP)SA(B) SE(I)SA(PB) SA(IPB) SE(IP) SE(I)SA(P) SA(IP) Encoder IDCT Method Average PSNR (db) SE(IPB) SE(IP)SA(B) SE(I)SA(PB) SA(IPB) SE(IP) SE(I)SA(P) SA(IP) Encoder IDCT Method (a) (b) Average PSNR (db) Decoder IDCT Method = S E(I)S A(PB) SE(IPB) SE(IP)SA(B) SE(I)SA(PB) SA(IPB) SE(IP) SE(I)SA(P) SA(IP) Encoder IDCT Method Average PSNR (db) Decoder IDCT Method = SA(IPB) SE(IPB) SE(IP)SA(B) SE(I)SA(PB) SA(IPB) SE(IP) SE(I)SA(P) SA(IP) Encoder IDCT Method (c) (d) Figure 5.4. Quality results for possible one-way communication system configurations quality is the same no matter which encoder DCT method is used. Thus, in the cases where SE(I)SA(PB) or SA(IPB) is applied at the decoder, using SE(IP) for the encoder IDCT (the conventional implementation) is a poor choice. Instead, if the decoder IDCT method is SE(I)SA(PB), the encoder IDCT should be SE(I)SA(P). Whereas, if the decoder IDCT method is SA(IPB), the encoder IDCT should be SA(IP). The explanation for this behavior is actually rather obvious. When the decoder reconstructs a predicted macroblock at the decoder (as in the case of P and B frames), it adds the received residual macroblock to a reference macroblock. This reference macroblock is from a previously decoded frame. The quality of the reconstruction depends partly on the difference between the encoder and decoder IDCT methods used to form the corresponding reference macroblock in the 20

21 Enc DCT Enc IDCT Dec IDCT Avg PSNR Reduction Decoder IDCT Energy % Reduction SE(IPB) SE(IP) SE(IPB) % SE(IPB) SE(IP) SE(IP)SA(B) % SE(IPB) SE(I)SA(P) SE(I)SA(PB) % Table 5.2. Quality/energy tradeoff of best configurations when concerned with decoder energy only encoder and decoder. If the IDCT methods are the same, the reference macroblock in the encoder and decoder will be exactly the same. If they are not the same, the reference macroblock at the decoder will be different from the encoder version, and this error will reduce the quality of the reconstructed macroblock. This type of error can cause even more problems when the reconstructed macroblock is part of a P frame, because the error may be propagated into other frames that reference the P frame. Therefore, to avoid these errors encoder and decoder IDCTs should match if possible. The configurations shown in Table 5.2 are the best quality/energy tradeoffs available in this case. The difference between these configurations and those of Table 5.1 is that 25.8% decoder IDCT energy reduction can be achieved at the expense of only 0.78 db average PSNR, rather than 1.46 db. This significant quality savings is a direct result of matching encoder and decoder IDCTs. An important result revealed when determining which configurations belong in Table 5.2 is that, at first glance, it appears that energy reduction of 27.7% can be achieved for the decoder IDCT at the expense of a relatively small average PSNR reduction of 1.06 db with the SE(IPB)/SA(IP)/SA(IPB) configuration. Such a result hints that the difference in quality between this method and SE(IPB)/SE(I)SA(P)/SE(I)SA(PB) might be negligible and allow a small amount of additional energy reduction. However, in this case, using the average PSNR quality measure is deceiving. If the PSNR for each frame of the sequence is considered instead, as is shown in Figure 5.5 for the Flower Garden sequence, a potentially serious problem can be identified. 21

22 PSNR (db) SE(IPB)/SE(I)SA(P)/SE(I)SA(PB) SE(I)SA(PB)/SE(IP)/SE(IPB) SE(IPB)/SA(IP)/SA(IPB) SA(IPB)/SE(IP)/SE(IPB) Frame Figure 5.5. Frame-by-frame quality comparison between configurations that use SA(IPB) and those that use SE(I)SA(PB) In Figure 5.5, it can be seen that all of these configurations have very similar quality for P frames and B frames. However, there is a sharp divergence in quality for I frames. SE(IPB)/SA(IP)/SA(IPB) and SA(IPB)/SE(IP)/SE(IPB) continue to have almost identical quality, whereas the other two algorithms, achieve 1 to 3 db better quality for I frames. Though SE(I)SA(PB)/SE(IP)/SE(IPB) can itself have I frame quality around 1 db less than SE(IPB)/SE(I)SA(P)/SE(I)SA(PB), this is not that serious. In this case, the B frames that precede and follow an I frame have nearly the same level of quality as the I frame, so it is difficult to perceive I frame degradation while viewing the video. The same is not true for the SE(IPB)/SA(IP)/SA(IPB) and SA(IPB)/SE(IP)/SE(IPB) configurations, where this artifact can potentially be seen by the viewer as momentary lapses in quality whenever I frames appear. This problem was verified during visual assessment of the four video test clips. Therefore, since this significant artifact has the potential to occur in general video coding, it is probably safest to avoid the methods that use SA(IPB) in both the encoder and decoder at the expense of a small amount of energy consumption. 22

23 Enc DCT Enc IDCT Dec IDCT Avg PSNR Reduction Encoder DCT+IDCT Energy % Reduction SE(IPB) SE(IP) SE(IPB) % SE(IP)SA(B) SE(IP) SE(IPB) % SE(I)SA(PB) SE(IP) SE(IPB) % SE(I)SA(PB) SE(I)SA(P) SE(IPB) % Table 5.3. Quality/energy tradeoff of best configurations when concerned with encoder energy only 5.3 Energy Sensitive Encoder, Energy Insensitive Decoder Another one-way communication system variation involves an energy sensitive encoder communicating with an energy insensitive decoder.an example of such an application is a wireless video camera that encodes video for later viewing on a tethered device. As before, the best quality/energy tradeoffs for this case are shown in Table 5.3. With these techniques, a small amount of video quality, 0.37 db average PSNR for the Flower Garden sequence, can be traded for a 13.8% reduction in energy consumption during DCT and IDCT computation in the encoder. For 0.78 db reduction, 19.4% energy reduction can be achieved. If energy consumption must be minimized further, a rather large quality reduction of 1.88 db can be traded for 24.9% energy reduction. The last option is somewhat interesting because it does not match the encoder IDCT with the decoder IDCT as discussed previously. It turns out that if decoder energy is not a concern, the best quality that can be achieved while using SE(I)SA(PB) for the encoder DCT and SE(I)SA(P) for the encoder IDCT is by using SE(IPB) for the decoder IDCT. This result occurs because the high quality of P and B frame decoding with the SE(IPB) IDCT in the decoder offsets errors due to reference macroblock mismatch that results when a P frame is referenced. A nice benefit of all the options in Table 5.3 is that they employ a decoder that uses no approximations. Thus, any standard MPEG-2 decoder can be used to achieve any of these tradeoffs. 23

24 Enc DCT Enc IDCT Dec IDCT Avg PSNR Reduction Encoder DCT+IDCT Energy % Reduction Decoder IDCT Energy % Reduction SE(IPB) SE(IP) SE(IPB) % % SE(IPB) SE(IP) SE(IP)SA(B) % % SE(IP)SA(B) SE(IP) SE(IPB) % % SE(IPB) SE(I)SA(P) SE(I)SA(PB) % % SE(I)SA(PB) SE(IP) SE(IPB) % % SE(IP)SA(B) SE(IP) SE(IP)SA(B) % % SE(IP)SA(B) SE(I)SA(P) SE(I)SA(PB) % % SE(I)SA(PB) SE(I)SA(P) SE(IPB) % % SE(I)SA(PB) SE(I)SA(P) SE(I)SA(PB) % % Table 5.4. Quality/energy tradeoff of best configurations when concerned with encoder and decoder energy 5.4 Energy Sensitive Encoder and Decoder A more general one-way communication case that actually encompasses the previous two cases utilizes an energy sensitive encoder and energy sensitive decoder. An example of such an application is a wireless video surveillance camera that encodes video and then transmits it to a wireless device at another location for viewing. The superior quality/energy tradeoffs shown in Table 5.4 for this case include those combinations chosen in Tables 5.2 and 5.3, as well as a variety of options that allow both the encoder and decoder to run with reduced energy at the same time. 5.5 Two-Way Communication with One Energy Sensitive Terminal Now, consider a system involving two-way communication between an energy sensitive terminal and energy insensitive terminal. In this configuration there are two video transmission channels and each terminal contains an encoder and decoder involved in separate channels. An example of such an application involves a wireless video conferencing appliance that receives video from and transmits video to a conference room with tethered equipment. The quality/energy tradeoffs shown in Table 5.5 are the best combinations for the forward (from wireless appliance to tethered equipment) and reverse (from tethered equipment to wireless appliance) channels, assuming it is desirable to keep the quality of both channels close to the same level. If 24

25 Forward Channel Enc DCT Forward Channel Enc IDCT Forward Reverse Channel Channel Dec IDCT Enc DCT Table 5.5. Quality/energy tradeoff of best configurations of a two-way communication system when concerned with energy of only one terminal significantly different levels of quality are desired for each channel for some reason, a number of other configurations are certainly available. With the options of Table 5.5, a small amount of video quality in both channels can be traded for a 15.8% reduction of DCT + IDCT energy consumption in the encoder and decoder of the wireless appliance. For the Flower Garden sequence, 0.26 db average PSNR is given up in the forward channel and 0.37dB is given up in the reverse channel. For 0.78 db quality reduction in the forward and reverse channels, 22.1% energy reduction can be achieved. Finally, for an additional 1.10 db reduction in the forward channel only, 25.3% energy reduction can be achieved. However, this may be too much quality to give up in the forward channel to reduce energy by only 3.2% more. Forward channel quality can be significantly worse than reverse channel quality for this case. Reverse Channel Enc IDCT Reverse Channel Dec IDCT Forward Reverse Channel Channel Avg PSNR Reduction Avg PSNR Reduction Sensitive Codec DCT+IDCT Energy % Reduction SE(IPB) SE(IP) SE(IPB) SE(IPB) SE(IP) SE(IPB) % SE(IP)SA(B) SE(IP) SE(IPB) SE(IPB) SE(IP) SE(IP)SA(B) % SE(I)SA(PB) SE(IP) SE(IPB) SE(IPB) SE(I)SA(P) SE(I)SA(PB) % SE(I)SA(PB) SE(I)SA(P) SE(IPB) SE(IPB) SE(I)SA(P) SE(I)SA(PB) % 5.6 Two-Way Communication with Two Energy Sensitive Terminals Finally, consider a system involving two-way communication between two energy sensitive terminals. In this configuration there are once again two video transmission channels and each terminal contains an encoder and decoder involved in separate channels. An example of such an application is two wireless video phones communicating with one another. The configurations shown in Table 5.6 are the best options available assuming it is desired that the quality of both channels be the same. This assumption is made to simplify the example. However, options with different quality reduction in each channel and different energy reduction in each terminal could certainly be supported if desired. 25

26 Enc DCT Enc IDCT Dec IDCT Avg PSNR Reduction Codec DCT+IDCT Energy % Reduction SE(IPB) SE(IP) SE(IPB) % SE(IP)SA(B) SE(IP) SE(IPB) % SE(IPB) SE(I)SA(P) SE(I)SA(PB) % SE(IP)SA(B) SE(I)SA(P) SE(I)SA(PB) % SE(I)SA(PB) SE(I)SA(P) SE(I)SA(PB) % Table 5.6. Quality/energy tradeoff of best configurations when concerned with energy of both communicating codecs In Table 5.6, the configuration and quality, in terms of average PSNR, for one channel is shown, as well as the relative energy consumption for all DCT and IDCT computation in one device. This data is identical for both channels and both devices in the system. With these techniques, a small amount of video quality, 0.37 db average PSNR in both channels for the Flower Garden sequence, can be traded for a 7.9% reduction in energy consumption during DCT and IDCT computation in the encoder and decoder of both devices. For an additional 0.41 db reduction, 14.2% energy reduction can be achieved. For an additional 0.66 db reduction, 22.1% energy reduction can be achieved. Finally, for an additional 0.75 db reduction, 25.3% energy reduction can be achieved. However, as in previous cases, this is likely to be too much quality to sacrifice in both channels to reduce energy consumption by only 3.2% more. 6 CONCLUSIONS The new approach presented here for MPEG-2 video shows the great potential datadependent quality/energy tradeoff techniques have for applications requiring energy reduction scalability or simply fixed energy reduction at the expense of a small amount of quality. The new approach boils down to choosing, in a data-dependent manner, which video frames should employ an approximate DCT/IDCT and which frames should use an exact DCT/IDCT during MPEG-2 encoding and decoding to achieve a desired quality/energy tradeoff. By studying six practical communication system examples through experimentation, a number of important issues were raised about applying the approach. For example, the type of 26

27 rate control used by the MPEG-2 encoder can have a significant effect on the way in which quality reduction is distributed across the different frame types. Encoder and decoder IDCT matching is important to avoid additional quality degradation. Looking at frame to frame quality is important to avoid tradeoff options that degrade certain frames much more than others. More importantly, the communication system examples verified the potential of the new approach over a wide range of new energy sensitive applications. As for future research directions, other DCT/IDCT approximations and algorithm mixing methods may provide better or wider ranging quality/energy tradeoffs for certain applications. Thus, they are good subjects for future work. By also designing other parts of an MPEG-2 codec to trade significant energy for small amounts of quality, a very effective energy scalable or fixed low energy codec could result for use in the applications considered in this paper. REFERENCES [1] A. Chandrakasan and R. Brodersen, ``Minimizing power consumption in digital CMOS circuits," Proc. of IEEE, vol. 83, pp , Apr [2] A. Chandrakasan, M. Potkonjak, R. Mehra, J. Rabaey, and R. Broderson, "Optimizing power using transformations," IEEE Trans. on Computer-Aided Design, vol. 14, pp , Jan [3] D. Singh et al., Power Conscious CAD Tools and Methodologies: A Perspective, Proc. of IEEE, 83(4), pp , April [4] E. Tsern and T. Meng, A low-power videorate pyramid VQ decoder, IEEE J. of Solid State Circuits, vol. 31, no. 11, pp , Nov [5] H. Samueli, An improved search algorithm for the design of multiplierless FIR filters with powers-of- two coefficients, IEEE Trans. on Circuits and Systems, pp , July [6] N. Sankaryya, K. Roy, and D. Bhattacharya, Algorithms for Low Power and High Speed FIR Filter Realization using Differential Coefficients, IEEE Trans. on Circuits and Systems-II, pp , June [7] M. Mehendale, S. Sherlekar, and G. Ventakesh, Low Power Realizations of FIR Filters for Programmable DSPs, IEEE Trans. on VLSI Systems, vol. 6, no. 4, pp , Dec