Specification Set-Top Box Video Quality Test Solution An Integrated Test Solution for IPTV Set-Top Boxes (over DSL) In the highly competitive telecom market, providing a high-quality video service is crucial for the commercial success of any triple-play strategy. The Set-Top Box (STB) testing solutions by AIMS (INT) gives the user the ability to run performance comparison tests on STB designs under laboratory controlled test conditions. Main features: Full reference video testing it compares the degraded video with a reference video sequence. Reliable objective video Quality of Experience (QoE) testing for Set Top Boxes (STBs). Correlation between the video mean opinion score (MOS) and subjective QoE tests. High flexibility and compatibility with different STBs and transmission methods. Option for testing IPTV over DSL is provided, where the DSL link the subjected to different line conditions and environmental effects including crosstalk, impulse noise (SHINE) and repetitive impulse noise (REIN). Automatic test solutions for batch capture, batch measurement, continuous capture/measurement. Reference video Impaired video Video Quality Estimation Software Video Quality Mean Opinion Score Figure 1: Video quality testing with a reference video.
STB Video Quality Testing Standalone Set-Top Box Testing Solution This solution allows the testing of Set-top Boxes (STBs) in various scenarios by measuring the video quality of experience (QoE) as perceived by an end user. A test video sequence, which is pre-degraded to emulate different IP network and transmission link conditions, is streamed directly into the Set-Top Box (Figure 2). The STB video output is captured and compared to the reference video to obtain a video mean opinion score (MOS), which correlates very well with subjective video QoE as perceived by the end user. This video test solution supports Standard Definition (SD) TV format and High Definition (HDMI) TV format. It is also independent of the used video compression technology, which is only limited by the STB under test. Possible compression standards include MPEG-2, MPEG4 Part 2 / 10, and VC-1. The test video sequence can also be streamed via different transmission media, such as DSL, coaxial cables, microwave, satellite, fibre optics, or any other suitable communications system. The solution is therefore highly flexible and can be used to test a wide range of IPTV deployment scenarios. Set-Top Box under test Test IP Video Stream MPEG2; MPEG4 Part 2/10; VC-1 Set Top-Box output Analogue: Component RGB / YUV; Composite PAL / NTSC Digital: HDMI, DVI IPTV streaming Video capture Video Quality Test IPTV streaming Video capture DSLAM Control Figure 2: Video quality testing of IPTV Set-Top Boxes standalone solution.
STB Video Quality Testing IPTV over DSL Testing Solution AIMS (INT) also offers a fully integrated IPTV over DSL video quality testing solution. In this set-up, the test IP video sequence is streamed to the Set-Top box via a complete DSL link (Figure 3). The STB output is then captured and compared to a reference video to obtain a video MOS score which matches the quality of experience of the end user. The link can be provided by the specialised AIMS DSLAM Simulator and uses an external noise generator and line simulator to reflect the impairments on a real DSL line, such as crosstalk, impulse noise, REIN, SHINE, ICE etc. Test software with a powerful but easy-to-use graphical user interface (GUI) can control the DSL connection, the line length and line noise. Test profiles are provided for an easy set-up of the DSL link according to the DSL Forum test specifications, and it is also possible to define custom profiles to match your needs. The test setup allows the testing of both IPTV systems with separate DSL modem and Set-Top box and devices which combine a DSL modem and STB in a single box. Noise and Line impairments Set-Top Box under test DSLAM Simulator DSL modem Received IP Video Stream MPEG2; MPEG4 Part 2/10; VC-1 Set Top-Box output Analogue: Component RGB / YUV; Composite PAL / NTSC Digital: HDMI, DVI Transmitted IP Video Stream MPEG2; MPEG4 Part 2/10; VC-1 IPTV streaming Video capture Video Quality Test IPTV streaming Video capture DSLAM Control Figure 3: Video quality testing of Set-Top Boxes for IPTV over DSL.
The STB VQM Tester The STB VQM Tester consists of an integrated solution of hardware and software which can capture video streams via a capture card, measure the QoE by comparing the captured streams with a reference stream, and display the video quality measurement results. This is all controlled by a software GUI (see picture below). Video capture manual start and stop of the capture process, or automatic capture of a set duration can be selected. The various options of the capture card can be set up via the button Setup Capture Card. Video quality measurement the quality of the captured video stream can be measured using a J.144 compliant algorithm or associated reduced reference algorithms. The measurement produces a mean opinion score (MOS) with a high correlation with subjective user Quality of Experience. The video quality measurement algorithm operates by providing a single measurement for a video sequence of up to 15 seconds. As well as providing additional parameters which identify some of the causes of the degradation in the tested video, (e.g. blurring, tiling, noise and errors.)
STB VQM Tester - Video Capture The STB VQM Tester software controls the capture card hardware via Microsoft DirectShow API support. A choice of capture cards for Standard Definition or High Definition video is available. A drop down menu allows selection of the capture card. Video Capture Device Selection The capture card video stream settings can be configured, to match the stream being output by the STB (E.g. PAL 8-bit 4:2:2) Full support for different STB output settings is provided, e.g. Set-up of the HDMI input port of the capture card)
STB VQM Tester - Video Quality Measurement The STB Tester includes a set of video quality algorithms based on the ITU Recommendation J.144. The LB Algorithm and Fast LB Algorithm are enhanced to perform well on videos degraded by both compression and transmission errors. The algorithms are listed on the STB VQM Tester specification sheet. Menu selection of measurement algorithm. The MOS Algorithms can be used to measure a video sequence which has been captured from a Standard Definition or High Definition video stream. The measurement capability is illustrated in the tables in the STB VQM Tester Specification sheets. CLI Command Set & Post Processing Tools A full CLI command set gives users the ability to write scripts to control the STB VQM Tester. This can be used to automate the capture of a number of video sequences and also automate the post processing of the captured sequences. Post Processing Tools can be used to schedule the processing of the captured files during non-work time e.g. night time or weekends. This maximises the useful time the tester has for capturing video sequences under various test scenarios.
Automated Capture and Measurement AIMS has developed specialised software tools to provide an automated testing solution in addition to the STB VQM Tester manual GUI. The VQM CLI server and STB VQM Sequencer have been developed for easy automatic and batch testing. The unit can be used to test a STB over several days continuously. This allows STB manufacturers to test the overall robustness and reliability of their product. The VQM STB CLI server has been produced with a fully documented CLI command set. This can allow users to produce their own test scripts, using any modern day scripting language. The GUI is as shown below:-
Automated Capture and Measurement The VQM STB Sequencer has been developed to provide a fully customizable solution without any need for the user to develop a control script. This GUI permits the user to set up a batch capture, to be followed by a batch measurement or alternatively set the tester to be set for continuous processing i.e. capture followed by measurement repeated until a user defined date and time. All results captured are available in.csv format. The GUI is as shown below:-
STB VQM Tester Specifications Capture Card Software: Video Formats: PAL 625 lines 720 x 576, 25 fps interleaved NTSC 525 lines 720 x 486, 29.97 fps interleaved 720HD 1080HD Analogue Video Interface Component RGB / YUV S-video PAL / NTSC Composite PAL / NTSC Connectors: SCART/Peritel, RCA, S-Video or BNC Digital Video Interface HDMI DVI Video Capture: Manual Video Capture Capture configuration resolution, frame rate Test Video Stream Formats: MPEG-2 MPEG4 Part 2/10 VC-1 Control & Post Processing Tools: STB_Tester Sequencer Custom Processing Reports in CSV format CLI Server Remote control Customised Scripts Measurement Software: VQM Algorithms ITU J.144 Full reference LB (Lowbandwidth) Reduced reference Fast LB Reduced reference VQM Calibrations ITU J.144 ITU J.144 - Time & region only RR Calibration RR Calibration & Scaling RR2 Calibration RR2 Calibration & Scaling RR2 Calibration Time & region only. Measured Quality/RCA Parameters: Mean Opinion Score HV Blurring Tiling Blurring Sharpening Colour Comb Noise Error Hardware Specification: Capture card options: SD Capture Card HD Capture Card PC : HP XW8600 (or equivalent) 4GB RAM 500GB Harddisk Keyboard & mouse Operating System: Windows Vista Bus (64bit) (No monitor included)
STB VQM Tester Software Specifications - Algorithms A. Video Scan Types: 1. Progressive - progressive scan videos (usually high definition-hd). 2. Interlaced upper field first interlaced videos for which the upper field is sent first (usually, but not always PAL) 3. Interlaced lower field first - interlaced videos for which the lower field is sent first (usually, but not always NTSC) B. Calibration method: 1. None no calibration performed. 2. ITU J.144 calibration according to ITU Recommendation J.144. 3. ITU J.144 - time & region only calibration according to ITU Recommendation J.144, but with time & region calibration only. 4. RR Calibration an improved calibration method. 5. RR Calibration & Scaling an improved calibration method with scaling. 6. RR2 Calibration an improved calibration method v.2 7. RR2 Calibration & Scaling an improved calibration method and scaling v.2 8. RR2 Calibration - time & region only - an improved calibration method v.2, where only the time and region are calibrated 9. Manual manually loading calibration. This method is not recommended, expert use only. C. MOS Algorithm: 1. None no MOS measurement is carried out. 2. ITU J.144 MOS Measurement according to ITU Recommendation J.144. 3. LB Algorithm an improved MOS Measurement algorithm which performs well on videos degraded by both compression errors and transmission errors. 4. Fast LB Algorithm a fast version of the improved MOS Measurement algorithm.
STB VQM Tester Software Specifications Algorithms Standard Definition Calibrations 1 2 3 4 5 6 7 8 9 MOS Algorithm ITU J.144 Low Bandwidth Fast Low Bandwidth Scan Type Progressive Field Field Progressive Field Field Progressive Field Field HDMI MOS Algorithm ITU J.144 Low Bandwidth Fast Low Bandwidth Calibrations 1 2 3 4 5 6 7 8 9 Scan Type Progressive Field Field Progressive Field Field Progressive Field