1 Quality of Experience in Internet television Stereoscopic How can we receive feedback to improve the QoE in an Internet television service? Mathias Lervold (1), mathias.lervold@gmail.com Liyuan Xing (2) Andrew Perkis (2) Liyuan Xing Centre for Quantifiable Quality of Service in Communication Systems Norwegian University of Science and Technology (1) Accenture (2) Centre for Quantifiable Quality of Service in Communication Systems (Q2S) Norwegian University of Science and Technology (NTNU), Trondheim, Norway VPQM, Scottsdale AZ, 13. 15. january 2010
2 Introduction Measuring perceptual quality in Internet television How can we receive feedback to improve the QoE in an Internet television service? VPQM, Scottsdale AZ, 13. 15. january 2010
3 Agenda Classification of QoE artifacts in Internet television Subjective tests Key findings VPQM, Scottsdale AZ, 13. 15. january 2010
4 Quality of Experience in Internet television Internet television is TV over the open Internet Quality challenges: Bandwidth restriction No guarantee of QoS Little control of the user end VPQM, Scottsdale AZ, 13. 15. january 2010
5 QoE artifacts classification in relation to service delivery VPQM, Scottsdale AZ, 13. 15. january 2010
6 Subjective tests to determine the QoE was run over 6 weeks Tests were run in-service on VG LIVE, a live streaming service of Norwegian football 18 users: 14 male / 4 female, 18-55 years old Similar to Single Stimulus Continuous Quality Evaluation Feedback through survey VPQM, Scottsdale AZ, 13. 15. january 2010
7 The user was asked to rate the VG LIVE service related to various aspects of the QoE Overall quality Adapting period Instant start/no buffering Video and audio flow Coding artifacts: Blurring Edge ringing Color bleeding Motion blurring VPQM, Scottsdale AZ, 13. 15. january 2010
8 Key finding 1: the QoE of Internet television is better than that of Internet video, but worse than that of broadcast TV VPQM, Scottsdale AZ, 13. 15. january 2010
9 Key finding 2: the results indicate that content, as well as context, can influence the QoE VPQM, Scottsdale AZ, 13. 15. january 2010
10 Key finding 3: service providers can receive feedback on the QoE in 4 stages 1. Full reference objective quality assurance at headend 2. Monitoring the network using bit rate parameters from the client 3. Extended user profile to include content/context information 4. QoE tool in the client for real time feedback, support and customization VPQM, Scottsdale AZ, 13. 15. january 2010
11 Summary A classification of QoE artifacts related to the service delivery of Internet television. Subjective tests show there is a need for improving the QoE of Internet television. A 4-stage QoE assessment plan from headend to user end can give the service provider a good measurment of the QoE for each individual user. Psychological factors such as content and context will need to be further researched in order to assess its impact on the QoE. VPQM, Scottsdale AZ, 13. 15. january 2010
12 Questions? VPQM, Scottsdale AZ, 13. 15. january 2010
13 Appendix
14 Internet television differs from IPTV in network openness, and from Internet video in production quality and business model
15 The service delivery of VG LIVE using streaming technology from Move Networks
16 The proposed QoE menu with 3 main features: QoE guide (1), support and user feedback (2), options and personalization (3).
17 Demographical/psychographical data 1
18 Demographical/psychographical data 2
19 User equipment 1
20 User equipment 2
21 Quality analysis 1: Quality comparison
22 Quality analysis 2: Details and sharpness
23 Quality analysis 3: Adaptation time
24 Quality analysis 4: Response time
25 Quality analysis 5: Audio/video smootheness
26 Quality analysis 6: Audio/video freeze/blackout
27 Post-survey 1: Overall impression
28 Post-survey 2: Function rating
29 Post-survey 3: Adaptive streaming and quality variations
30 Post-survey 4: Value of the service