A Backlight Optimizatio Scheme for Playback o Mobile Devices Liag Cheg, Shivajit Mohapatra, Magda El Zarki, Nikil Dutt ad Nalii Vekatasubramaia Doald Bre School of Iformatio ad Computer Scieces Uiversity of Califoria, Irvie Irvie, CA 92697 Email: (lcheg61, mopy, magda, dutt, alii)@ics.uci.edu Abstract For a typical portable hadheld device, the backlight accouts for a sigificat percetage of the total eergy cosumptio (e.g., aroud 3% for a Compaq ipaq 365). Substatial eergy savigs ca be achieved by dyamically adaptig backlight itesity levels o such lowpower portable devices. I this paper, we aalyze the characteristics of video streamig services ad propose a adaptive scheme called Quality Adapted Backlight Scalig (QABS), to achieve backlight eergy savigs for video playback applicatios o hadheld devices. Specifically, we preset a fast algorithm to optimize backlight dimmig while keepig the degradatio i image quality to a miimum so that the overall service quality is close to a specified threshold. Additioally, we propose two effective techiques to prevet frequet backlight switchig, which egatively affects user perceptio of video. Our iitial experimetal results idicate that the eergy used for backlight is sigificatly reduced, while the desired quality is satisfied. The proposed algorithms ca be realized i real time. I. INTRODUCTION With the widespread availability of 3G cellular etworks, mobile had-held devices are icreasigly beig desiged to support streamig video cotet. These devices have striget power costraits because they use batteries with fiite lifetime. O the other had, multimedia services are kow to be very resource itesive ad ted to exhaust battery resources quickly. Therefore, coservig power to prolog battery life is a importat research problem that eeds to be addressed, specifically for video streamig applicatios o mobile hadheld devices. Most had-held devices are equipped with a TFT (Thi-Film Trasistor) LCD (Liquid Crystal Display). For these devices, the display uit is drive by the illumiatio of backlight. The backlight cosumes a cosiderable percetage of the total eergy usage of the hadheld device; it cosumes 2%-4% of the total system power (for Compaq ipaq) [1]. Dyamically dimmig the backlight is cosidered a effective method to save eergy [1], [2], [3] with scalig up of the pixel lumiace to compesate for the reduced fidelity. The lumiace scalig, however, teds to saturate the bright part of the picture, thereby affectig the fidelity of the video quality. I [2], a dyamic backlight lumiace scalig (DLS) scheme is proposed. Based o differet scearios, three compesatio strategies are discussed, i.e., brightess compesatio, image ehacemet, ad cotext processig. However, their calculatio of the distortio does ot cosider the fact that the clipped pixel values do ot cotribute equally to the quality distortio. I [3], a similar method, amed cocurret brightess ad cotrast scalig (CBCS), is proposed. CBCS aims at coservig power by reducig the backlight illumiatio while retaiig the image fidelity through preservatio of the image cotrast. Their distortio defiitio ad proposed compesatio techique may be good for static image based applicatios, such as the graphic user iterface (GUI) ad maps, but might ot be suitable for streamig video scearios, because their cotrast compesatio further compromises the fidelity of the images. I additio, either [2] or [3] solves the problem associated with frequet backlight switchig which ca be quite distractig to the ed user. I this paper, we explicitly icorporate video quality ito the backlight switchig strategy ad propose a quality adaptive backlight scalig (QABS) scheme. The backlight dimmig affects the brightess of the video. Therefore, we oly cosider the lumiace compesatio such that the lost brightess ca be restored. The lumiace compesatio, however, ievitably results i quality distortio. For the video streamig applicatio, the quality is ormally defied as the resemblace betwee the origial ad processed video. Hece, for the sake of simplicity ad without loss of geerality, we defie the quality distortio fuctio as the mea square error (MSE)(see Equatio (1)) ad the quality fuctio as the peak sigal to oise ratio (PSNR)(see Equatio (2)), both of which are well accepted objective video quality measuremets. MSE = 1 M M (x i y i) 2 (1) i=1 = 1log 1 M i=1 255 2 (x i y i) 2 (2) where x i ad y i are the origial pixel value ad the recostructed pixel value, respectively. M is the umber of pixels per frame. It is to be oted that ay improved quality metrics may be adopted to replace the MSE/PSNR metrics used here without affectig the validity of our proposed scheme. As is metioed i [3], for video applicatios, the cotiuous chage i the backlight factor will itroduce iter-frame brightess distortio to the observer. I our experimets, we fid that the uecessary backlight chages fall ito two categories: (1) small cotiuous chages over adjacet frames; (2) abrupt huge chages over a short period. Therefore, we propose to quatize the calculated backlight to elimiate the small cotiuous chage ad use a low-pass digital filter to smooth the abrupt chages. The rest of the paper is orgaized as follows. I Sectio II, we itroduce the priciple of the LCD display - experimetal results show that backlight dimmig saves eergy while the pixel lumiace compesatio results i miimal overhead. I Sectio III, we preset our QABS scheme, which icludes determiig the backlight dimmig factor ad two supplemetary methods to avoid excessive backlight switchig. Sectio IV shows our prototype implemetatio, experimetal methodology ad simulatio results. We coclude our work i Sectio V. II. CHARACTERISTICS OF LCD I this sectio, we outlie the characteristics of the LCD uit from two perspectives, the LCD display mechaism ad the LCD power cosumptio, both of which form the basis for our system desig.
25 25 Frequecy 15 Frequecy 15 5 5 5 1 15 2 25 3 Lumiace value 5 1 15 2 25 3 Lumiace value (a) Origial image (b) Compesated image (c) Histogram before clippig (d) Histogram after clippig Fig. 1. Image ad its lumiace histogram before ad after clippig A. LCD Display The LCD pael does ot illumiate itself, but displays by filterig the light source from the back of the LCD pael [2][3]. There are three kids of TFT LCD paels: trasmissive LCD, reflective LCD, ad trasflective LCD. We focus i this paper o the reflective, sice it is the most commoly used LCD for hadheld devices. Heceforth, whe we metio LCD, we refer to reflective LCD ad we refer to both backlight ad forelight as backlight. As will be show, our idea is geeric to ay backlight based LCD. The perceptual lumiace itesity of the LCD display is determied by two compoets: backlight brightess ad the pixel lumiace. The pixel lumiace ca be adjusted by cotrollig the light passig through the TFT array substrate. Users may detect a chage i the display lumiace itesity if either of these two compoets is adjusted. That is, the backlight brightess ad the pixel lumiace ca compesate each other. I Sectio II-B, we will show that the pixel lumiace does ot have a oticeable impact o the eergy cosumptio, whereas the backlight illumiatio results i high eergy cosumptio. Hece, i geeral, dimmig backlight level while compesatig the pixel lumiace is a effective way to coserve battery power i had-held devices. Let the backlight brightess level ad the pixel lumiace value be L ad Y, respectively, ad the perceived display lumiace itesity I. We may deote I usig Equatio (3). I = ρ L Y (3) where ρ is a costat ratio, deotig the trasmittace attribute of the LCD pael, ad as such ρ Y is the trasmittace of the pixel lumiace. We may reduce the backlight level to L by multiplyig L with a dimmig factor α, i.e., L = L α, < α < 1. To maitai the overall display lumiace I ivariable, we eed to boost the lumiace of the pixel to Y. Sice the pixel lumiace value is ormally restricted by the umber of bits that represet it (deoted as ), Y may be clipped if the origial value of Y is too high or the α is too low. The compesatio of the backlight is described i Equatio (4). { Y/α, if Y < α 2 Y = 2, if Y (α 2 ) Combiig Equatio (4) ad Equatio (3), we have { I, if Y < α 2 I = ρ L α 2 if Y (α 2 ) (4) (5) Equatio (5) clearly shows that the perceived display itesity may ot be fully recovered, istead, it is clipped to ρ L α 2 if Y (α 2 ). I Figure 2, we illustrate the clippig effect of the display lumiace. I Figure 1-a ad Figure 1-c, we show a image ad its lumiace histogram. This image is the first frame of a typical ews video clip ( ABC eye witess ews ) captured from broadcastig TV sigal. Figure 1-b ad Figure 1-d illustrate the image ad its lumiace histogram after backlight dimmig ad pixel lumiace compesatio. Figure 1-d shows that the pixels with lumiace higher tha 156 are all clipped to 156. This clippig effect elimiates the variety i the bright areas, which is subjectively perceived as the lumiace saturatio ad is objectively assessed as 3dB with referece to the origial image show i Figure 1-a. B. LCD Power Model I our experimets, we observe that the backlight dimmig ca save eergy whereas the compesatio process, i.e., scalig up the lumiace of the pixel, has a egligible eergy overhead. We measure the eergy savig as a differece of the total system power cosumptio with backlight set to differet levels from that with the backlight tured to the maximum (brightest). Figure 3 shows the plot betwee the various backlight levels ad their correspodig eergy cosumptio for a Compaq ipaq 365 ruig Liux. A more detailed setup of our experimets is described i Sectio IV. It is oticed that the backlight eergy savig is almost liear to the backlight level ad ca be estimated usig Equatio (6). y = a1 x + a2 (6) where y is the eergy savigs i Watt; x deotes the backlight level; a1 ad a2 are coefficiets. We apply the curve fittig fuctio of MATLAB ad obtai a1 =.29567 ad a2 =.73757 with the largest residual fittig error as.85731. Cotrary to the backlight switchig, the pixel lumiace scalig is ucorrelated to the eergy cosumptio. I Figure 4, we show that for oe specified backlight level (BL) the system eergy cosumptio basically remais stable ad is idepedet of the lumiace scalig. Figure 3 ad Figure 4 justify the validity of the geeric backlight power coservatio approach, i.e., dimmig the backlight while ehacig the pixel lumiace value. Note that i Figure 4, BL refers to the backlight level ad Lumiosity Scalig Factor refers to α. I the ext sectio, we apply this method to the video streamig sceario, discussig a practical scheme to optimize the
I' ( ρ L 2 α) Power Savig (W).8.7.6.5.4.3.2.1 Measured Estimated Power (W) 2.8 2.7 2.6 2.5 2.4 2.3 2.2 2.1 2 1.9 BL=255 BL=25 BL=155 BL=15 BL=55 BL= Distortio (MSE) 1.8 1.6 1.4 1.2 1.8.6.4.2 2 x 16 Fig. 2. ( 2 α) Clippig Y 5 1 15 2 25 Backlight Level Fig. 3. Power savig vs. backlight level 1.8.1.3.5.7.9 1.1 1.3 1.5 1.7 1.9 2.1 2.3 Lumiosity Scalig Factor Fig. 4. Eergy overhead 5 1 15 2 25 3 Backlight level (Alfa) Fig. 5. Alfa MSE with differet backlight dimmig while takig ito cosideratio the effect o video distortio. III. ADAPTIVE BACKLIGHT SCALING As explaied i Equatio (5), the backlight scalig with the lumiace compesatio may result i quality distortio. The amout of backlight dimmig, therefore, has to be restricted such that the video fidelity will ot be seriously affected. A. Optimized Backlight Dimmig We defie the optimized backlight dimmig factor as the oe whose iduced distortio is closest to a specified threshold. Heceforth, we replace the factor α with the real backlight level Alfa, Alfa = N α (N is the umber of backlight levels (256 for Liux o ipaq)), ad the optimized backlight dimmig is represeted as Alfa. I Figure 5, we illustrate the image quality distortio i terms of MSE over differet backlight levels. (Note that we use the image show i Figure 1-a.) We see that as Alfa icreases, the iduced video quality distortio due to the brightess saturatio mootoously decreases. Hece, for a give distortio threshold, we ca fid a uique Alfa(= Alfa ) for each image. I video applicatios, for a give distortio, differet frames may have distict Alfa, depedig o the lumiace histogram of that frame. However, it is hard to have a accurate aalytical represetatio of the quality distortio usig Alfa as a parameter. We therefore adopt a optimized search based approach, where we calculate the MSE distortio with differet Alfa util the specified distortio threshold is met. The results of our scheme are accurate ad ca be used as the bechmark for the desig of other aalytical methods. Figure 6 shows the exhaustive searchig algorithm for fidig Alfa for oe image. FidAlfa(th) takes the distortio threshold (th) as iput, ad returs the Alfa as output. Note that MSE(Alfa) calculates the MSE with the specified Alfa for oe frame. However, the complexity of a exhaustive search show i Figure 6 is too high. As show i Equatio (2), the per-frame MSE calculatio cosists of M multiplicatios ad 2M additios. M is the umber of pixels i oe frame, e.g., M = 25344 for QCIF format video. We regard the per-frame MSE as the basic complexity measuremet uit. We assume that the optimized backlight level is uiformly distributed i [, N], ad thus the complexity of algorithm i Figure 6 is O(N). I our test, N = 256. Obviously, the optimized backlight dimmig factor ca hardly be calculated i real-time. Therefore, we apply a faster bisectio method [4] to improve the algorithm for fidig Alfa. Sice we ca easily fid a upper boud (deoted as u) ad a lower boud (deoted as d) o the backlight levels, we get as good a approximatio as we wat by usig bisectio. We assume that u > d ad let ǫ be the desired precisio ad preset the algorithm i Figure 7. By usig the bisectio method, we may achieve the complexity of O(log 2N) i the worst case. For istace, for N = 256 ad ǫ = 1, we oly eed to calculate per-frame MSE at most eight times, which is fast eough for real-time processig. Fig. 6. Fig. 7. Exhaustive algorithm for fidig Alfa Fast algorithm for fidig Alfa B. Smoothig the Backlight Switchig It has bee discussed i [3] that the backlight dimmig factor may chage sigificatly across cosecutive frames for most video applicatios. The frequet switchig of the backlight may itroduce a iter-frame brightess distortio to the observer. Hece, it is ecessary to reduce frequet backlight switchig. I our study, we observe that the calculated Alfa, although based o a idividual image, does ot experiece huge fluctuatios durig a video scee, i.e., a group of frames that are characterized with similar cotet. Actually, the redudacy amog adjacet frames costitutes the major differece betwee the video ad the static
image applicatio ad has log bee utilized to achieve higher compressio efficiecy. Hece, the backlight switchig should be smoothed out withi the scee ad most favorably oly happe at the boudary of video scees. We propose two supplemetary methods to smooth the acquired Alfa i the same video scee. First, we apply a low-pass digital filter to elimiate ay abrupt backlight switchig that is caused by the uexpected sharp lumiace chage. The passbad frequecy is determied by the subjective perceptio of the flicker momet ad the frame display rate. Secod, we propose to quatize the umber of backlight levels, i.e., ay backlight level betwee two quatizatio values ca be quatized to the closest level, by which we prevet the eedless backlight switchig for small lumiace fluctuatios durig oe scee. I our experimets, we quatize all 256 levels to N levels (N=5 i our study). We switch the backlight level oly if the calculated Alfa chages drastically eough, so that it falls ito aother quatized level. IV. PERFORMANCE EVALUATION I this sectio, we itroduce our prototype implemetatio, the methodology of our measuremet ad the performace of the proposed algorithm. A. Prototype Implemetatio Figure 8 shows a high level represetatio of our prototype system. Our implemetatio of the video streamig system cosists of a video server, a proxy server ad a mobile cliet. We assume that all commuicatio betwee the server ad the mobile cliet is routed through a proxy server typically located i proximity to the cliet. Exteral Voltage Supply ( 5V ) Server Trascoder Decoder QABS Quality ifo Ecoder Fig. 8. Proxy Server Backlight ifo. V R BNC-211 coector Fig. 9. V ipaq R=.22ohm Mutiplexer Comm. Maager Comm. Maager User quality preferece ipac Cliet Demutiplexer LCD Cotrol Module Pixel value Decoder LCD Display Backlight levels Backlight Adjustmet Module Prototype implemetatio. C ipaq video DAQ Board Wireless video Power measuremet system Setup for our measuremets. Proxy The video server is resposible for streamig compressed video to the cliet; The proxy server trascodes the received stream, adds the appropriate cotrol iformatio, ad relays the ewly formed stream to the mobile cliet (Compaq ipaq 365 i our case). For the sake of simplicity ad without loss of geerality, i our iitial prototype implemetatio, we use the proxy server to also double up as our video server. The proxy server icludes four primary compoets - the video trascoder, the proposed QABS module, the sigal multiplexer, ad the commuicatio maager. The trascoder ucompresses the origial video stream ad provides the pixel lumiace iformatio to Mea Variace Backlight Level 15 1 5 2 4 6 8 12 14 16 18 5 4 3 22 2 18 16 14 12 2 4 6 8 12 14 16 18 Fig. 1. Qua = 3dB Basic statistics of abc ews. 1 Qua = 35dB Qua = 4dB 8 2 4 6 8 12 14 16 18 Fig. 11. Alfa adapted to three give quality thresholds. the QABS module. The QABS module calculates the optimized backlight dimmig factor based o the user quality preferece feedback received from the cliet (user). The multiplexer is used to multiplex the optimized backlight dimmig iformatio with the video stream. The commuicatio maager is used to sed this aggregated stream to the cliet. O the mobile cliet, the demultiplexer is used to recover the origial video stream ad the ecoded backlight iformatio from the received stream. The LCD cotrol module reders the decoded image oto the LCD display. The backlight iformatio is fed to the Backlight Adjustmet Module, which cocurretly sets the backlight value for the LCD. I particular, users may sed the quality request to the proxy whe requestig for the video, based o his/her quality preferece as well as cocer for battery cosumptio. B. Measuremet Methodology For video quality ad power measuremets, we use the setup show i Figure 9. The proxy i our experimets is a Liux desktop with a 1GHz processor ad 512MB of RAM. All our measuremets are made o a Compaq ipaq 365. We use a Natioal Istrumets PCI DAQ board to sample voltage drops across a resistor ad the ipaq, ad sample the voltage at 2K samples/sec. We calculate the istataeous ad average power cosumptio of the ipaq usig the formula P ipaq = V R R V ipaq. C. Experimetal Results I our simulatio, we use a video sequece captured from a broadcasted ABC ews program, whose first frame is show i Figure 1-a. We choose this video as represetative of a typical usage of a PDA. I Figure 1, we show the basic statistics (i.e., the mea ad the variace of lumiace per frame) of this video. We assume that the users are give three quality optios, fair, good, ad excellet, which respectively correspod to the PSNR
value of 3dB, 35dB, ad 4dB. After applyig the algorithm Proc: FastFidAlfa, we obtai the adapted Alfa for these three quality prefereces, as is show i Figure 11. It ca be see that higher video quality eeds higher backlight level o average. I Figure 12, we show Alfa before ad after the backlight smoothig process. It is see that the small variatio ad the abrupt chage of the backlight switchig are sigificatly elimiated after the filterig ad quatizatio. I additio, as we expected, the backlight switchig mostly happes at the boudary of major scees. I Table I, we summarize the results of our QABS. The mea Alfa of differet quality prefereces produces a quality o average very close to the pre-determied quality threshold. It is oted that differet quality requiremets result i various power savig gais. Higher quality preferece must be traded usig more backlight eergy. Nevertheless, we ca still save 29% eergy that is supposed to be cosumed by the backlight uit if we set the quality preferece to be Excellet. Eergy Cosumed (mj) Total Eergy Cosumptio with optimizatio o optimizatio 18 16 14 1 8 6 4 # 1 13 25 37 49 511 613 715 817 919 121 1123 1225 1327 1429 1531 1633 1735 Fig. 14. Eergy cosumptio with ad without optimizatio for abc ews video clip. Eergy (mj) 4 35 3 25 15 5 with optimizatio o optimizatio # 1 19 37 55 73 91 19 127 145 163 181 199 217 235 253 271 289 37 325 343 361 379 397 Fig. 15. Eergy cosumptio with ad without optimizatio for forema video clip. backlight adaptatio. As see from the graph, the eergy savigs from our backlight adaptatio for the ABCNews clip is 35%-4% of the total eergy cosumed due to backlight. Eve for videos that offer very little opportuity to aggressively perform backlight adaptatio (e.g. forema video clip, which is simply a talkig head), we ca achieve eergy savigs as high as 14-2% without sacrificig video quality. Fig. 12. Alfa before ad after filterig ad quatizatio. 8 6 4 2 2 4 6 8 12 14 16 18 8 6 4 2 2 4 6 8 12 14 16 18 8 6 4 2 Fig. 13. After smoothig Before smoothig After smoothig Before smoothig After smoothig Before smoothig Quality threshold = 3dB Quality threshold = 35dB Quality threshod = 4dB 2 4 6 8 12 14 16 18 Quality before ad after Alfa smoothig. I Figure 13, we show that the filterig ad quatizatio process may lead to istataeous quality fluctuatio, which is cotrasted to the cosistet quality before backlight smoothig. Nevertheless, we observe that the quality fluctuatio is aroud the desigated quality threshold ad mostly happes at scee chages. I Figure 14 ad Figure 15, we compare the actual eergy cosumptio o a Compaq ipaq with ad without our quality aware V. CONCLUSION I this paper, we apply a backlight scalig techique to video streamig applicatios, ad explicitly associate backlight switchig to the perceptual video quality i terms of PSNR. The proposed adaptive algorithm is fast ad effective for reducig the eergy cosumptio while maitaiig the desigated video quality. To reduce the frequecy of backlight switchig, we propose two supplemetary schemes that smooth the backlight switch process such that the user perceptio of the video stream ca be substatially improved. VI. ACKNOWLEDGEMENT We would like to thak Stefao Bossi ad Michael Philpott who helped us i the simulatio ad the system setup. REFERENCES [1] S. Pasricha, M Luthra, S. Mohapatra, N. Dutt, N. Vekatasubramaia, Dyamic Backlight Adaptatio for Low Power Hadheld Devices, IEEE Desig ad Test (IEEE D&T), Special Issue o Embedded Systems for Real Time Embedded Systems, Sep. 24. [2] N. Chag, I. Choi, ad H. Shim, DLS: Dyamic Backlight Lumiace Scalig of Liquid Crystal Display, IEEE Trasactio o VLSI System, vol. 1, Aug. 24. [3] W.-C. Cheg, Y. Hou, ad M. Pedram, Power Miimizatio i a Backlit TFT-LCD Display by Cocurret Brightess ad Cotrast Scalig, Proceedigs of the Desig, Automatio ad Test i Europe, Feb. 24. [4] J. L. Zachary, Itroductio to Scietific Programmig: Computatioal Problem Solvig Usig Maple ad C. Telos Publishers, 1996. TABLE I RESULTS OF QABS (G: GOOD; F: FAIR; E: EXCELLENT) Alfa Mea Quality(dB) Power Savig(%) F G E F G E F G E 149 162 186 3.17 34.28 42.31 41.8% 36.7% 27.3%