Muldia Consumer Terminals and QoS A QoS approach for Muldia Consumer Terminals with media processing in software Reinder J. Bril r.j.bril@tue.nl http://www.win.tue.nl/~rbril/ Muldia Consumer Terminals /video: perception is key high volume electronics: cost-constrained requires average-case resource allocation High quality and video: have real- requirements Quality of Service (QoS) collective effort of service performances that determine the degree of satisfaction of the user of that service (International Telecommunications Union) 9-05-006 A QoS approach: V-QoS Aim: Cost-effective high-quality video processing in software for muldia consumer terminals Motivation: openness & flexibility Boundary condition: Cost-effectiveness Preserve existing system qualities Co-operative approach: Organizational (multi-disciplinary) System Quality of Service Resource Management (QoS-RM) V-QoS Multi-disciplinary QoS approach Scalable Video s (SVA) University of Madrid (dit/upm) University of Mannheim University of Illinois at Urbana- University of St. Petersburg Champaign (UIUC) ITEA/Europa, ITEA/Robocop, OZONE, Overview Muldia Consumer Terminals Muldia Consumer Terminals (Summary ) Media processing from dedicated HW to SW Platforms are resource constraint High quality video has real requirements A QoS approach (Summary ) Adaptive applications Budget-based resource manager Control hierarchy Acknowledgement & References RF Tuner DVB Tuner VGA Cable modem DVD CDx front end IEEE 9 interface CVBS interface YC interface Focus: Receivers in broad-cast environments High-quality video applications 5 (by courtesy of Maria Gabrani) 6
Traditional High-End TV Architecture Digital video platform RAM TXT acq control prog ROM Traditional TV sets and Set-Top Boxes: Expectations: HF electr. cable antenna 00Hz FALCON IC PALplusIC 6:9 helper standard PICNIC picture decoder enhancement control YUV YUV fh fh PAL/NTSC PiP standard + CVBS decoder Mem RGB fh RGB fh display Fixed algorithms for fixed HW architectures Upgrade for new services and applications is problematic Systems are not flexible VLIW Bus MIPS Coprocessors Memory Upgradeable for new services and applications Fast -to-market for new features Enabling approach for product families demod. Audio proces. demod NICAM decod. channel bits chan. decod. transp. demux MPEG MPEG video (by courtesy of Egbert G.T. Jaspers) 7 8 HW Architectures vs. SW Applications Flexibility Product Families SW-Modules Resources min max Midrange Highend Mode Mode Mode 9 0 Scalability Smooth mode transition Alg. Alg. Alg. Alg. Alg. Mode Mode Initial Transitory Final
Platform constraints Platform constraints Cost traditional systems scalable approach Quality traditional systems scalable approach Cost-effectiveness requirement High volume: low bill of material Low power Software solutions are relatively expensive (mm silicon / power) Average-case and worst-case are far apart target limit Functionality target limit Functionality Fluctuating resource requirements Fluctuating resource requirements load worst-case Temporal load changes (very frequent) from I frame to B frame (MPEG ) more or less motion transient high peaks Structural load changes (less frequent) scene change from movie to camera from news to commercial structural load running average temporal load 5 6 Platform constraints Cost-effectiveness requirement High volume: low bill of material Low power Software solutions are relatively expensive (mm silicon / power) Average-case and worst-case are far apart Conclusion: Aim for average-case resource allocation TV companies invest heavily in video enhancement, e.g. temporal up-scaling Input stream: Hz (movie) Rendered stream: 60 Hz (TV screen) original up-scaled 7 8
TV companies invest heavily in video enhancement, e.g. temporal up-scaling TV companies invest heavily in video enhancement, e.g. temporal up-scaling Input stream: Hz (movie) Input stream: Hz (movie) original original up-scaled up-scaled displayed displayed Deadline miss leads to wrong picture. Deadline misses tend to come in bursts (heavy load). Valuable work may be lost. 9 QoS trade-off (at run-): Lesser picture quality often better than temporal incorrectness. 0 Muldia Consumer Terminals - Summary QoS = degree of user satisfaction (reminder) User satisfaction has to do with perception: Lesser picture quality often better than temporal incorrectness. Quality fluctuations are perceived as non-quality. With a scene change, the brain needs some to re-focus. Most people focus on one thing at a (user focus) User focus normally is at the center of a window, the window that received the latest (remote) command. Only video specialists can make the necessary trade-offs. Media processing from dedicated HW to SW Flexibility & scalability Fast -to-market Product families Platform constraints Aim for average-case resource allocation High quality video Has real- requirements; is about perception -> QoS; real- is a QoS parameter; QoS is primarily an application-domain issue. Overview User applications (prototype) Muldia Consumer Terminals A QoS approach (Summary ) Adaptive applications Budget-based resource manager Control hierarchy Acknowledgement & References Input DVD main pip Output disk Media processing Modes mode : main mode : main + pip mode : main + pip + disk In mode, main + pip + disk can not run at highest quality.
Application execution model (prototype) Scalable DVD read digit dmux sharp enh. main: scalable scaler rend mixer signal in ALGORITHM FOR MEDIA PROCESSING FUNCTION FUNCTION FUNCTION signal out digitizer: non-scalable pip: scalable mixer : non-scalable hw enc. writer scaler FUNCTION Quality disk : non-scalable buffer enc. dec task data transfer scalable task connection to HW IO hierarchical task Both main and pip are scalable. 5 control signal for quality level QUALITY CONTROL Resource needs 6 QoS parameters for media processing Workstation in network env. Receiver in broadcast env. Resolution Negotiable (screen) Defined by standard (e.g. PAL, NTSC, ATSC, VGA) Window size Negotiable Fixed (main or PiP) or user determined Frame-rate Low (max 0 Hz) and variable High (50 0Hz) and fixed QoS parameters for media processing Parameters for MCTs: Number and dimensions of filter coefficients; Number of fields used for image processing; Reference objects (lines, points, blocks); Type of processing (linear, non-linear); Frame-skips Acceptable Low tolerance Compression quality and bit-rate Negotiable Determined by provider 7 8 Example: Sharpness Enhancement Resource range for SVA examples Sharpness Enhance. s s Detail Filter Noise Measurement Amplitude Control Filter AC NM QL0 No No No QL D No No QL D No No QL D Yes Yes 9 QL0 QL QL QL0 QL QL Qual. estimate Qual. estimate 0 0 0 0 Resource needs [MIPS] Down-Scaler 0 5 0 5 Resource needs [MIPS] 0 5
Budget-based resource manager Temporal and spatial isolation: Reserved, guaranteed, and enforced budgets e.g. 0% of the every 0 ms; MB of contiguous memory (size and granularity) Resource budgets are allocated to RCEs (Resource Consuming Entities) Budget-based resource management + RCE Temporal and spatial isolation: Robustness: Inter-RCE: budgets are guaranteed no interference. In case of a conflict, there is always a victim. Intra-RCE: In RCE has to get by with their budget, i.e. deal robustly with a local overload. MPEG read dmux sharp enh. rend app. budget main: scalable video RCEs run on a virtual platform app. budget Budget-based resource management + RCE Temporal independence (RCE): read dmux main: scalable scaler sharp enh. pip: scalable rend mixer mixer : non-scalable The two inputs of mixer must be explicitly uncoupled to provide temporal independence between main and pip. Budget-based resource management + RCE Two level scheduling: resource budgets admission test tasks within an RCE RCE: temporally independent robust in case of overload real- adapts to budget limitations Question: What about and video? Dynamic load (reminder) Control hierarchy Temporal load changes (very frequent) from I frame to B frame more or less motion transient high peaks Structural load changes (less frequent) scene change from movie to camera from news to commercial Mode change (infrequent) functional change external trigger Dynamic load at different -scales: Temporal load changes; Structural load changes; Mode change. Multiple layers of control, e.g. local quality control of applications temporal load changes; structural load changes. global system utility control structural load changes; mode changes. 5 6 6
Co-operative QoS approach A QoS approach - summary Control Hierarchy Global system utility control Adaptive applications Local quality control SVAs Resource manager Optimizes system utility, sets quality levels + allocates resources Provide quality levels + estimated resource req. Provides guaranteed resource budgets Adaptive applications scalable applications local quality control Budget-based resource manager reserved, guaranteed and enforced budgets providing temporal and spatial isolation Control hierarchy layers of control to handle dynamic load at different -scales 7 8 Acknowledgement V-QoS program: Christian Hentschel; Clara M. Otero Pérez; Wim F.J. Verhaegh; Clemens C. Wűst; Other program members and partners. References [Hentschel et al 0] C. Hentschel et al., Video Qualityof-Service for consumer terminals - A Novel system for programmable components, IEEE Transactions of Consumer Electronics (TCE), 9():67 77, November 00. [Isovíc and Fohler 0] D. Isovíc and G. Fohler, Quality aware MPEG- Stream Adaptation in Resource Constrained Systems, In: Proc. 6 th ECRTS, pp. -, 005. [Bril 0] R.J. Bril, Real- scheduling for media processing using conditionally guaranteed budgets, PhD thesis TU/e, IPA Dissertation Series 00, Sept. 00, http://alexandria.tue.nl/extra/009.pdf. 9 0 7