Stereoscopic. How can we receive feedback to improve the QoE in an Internet television service? Liyuan Xing

Transcription

1 1 Quality of Experience in Internet television Stereoscopic How can we receive feedback to improve the QoE in an Internet television service? Mathias Lervold (1), Liyuan Xing (2) Andrew Perkis (2) Liyuan Xing Centre for Quantifiable Quality of Service in Communication Systems Norwegian University of Science and Technology (1) Accenture (2) Centre for Quantifiable Quality of Service in Communication Systems (Q2S) Norwegian University of Science and Technology (NTNU), Trondheim, Norway VPQM, Scottsdale AZ, january 2010

2 2 Introduction Measuring perceptual quality in Internet television How can we receive feedback to improve the QoE in an Internet television service? VPQM, Scottsdale AZ, january 2010

3 3 Agenda Classification of QoE artifacts in Internet television Subjective tests Key findings VPQM, Scottsdale AZ, january 2010

4 4 Quality of Experience in Internet television Internet television is TV over the open Internet Quality challenges: Bandwidth restriction No guarantee of QoS Little control of the user end VPQM, Scottsdale AZ, january 2010

5 5 QoE artifacts classification in relation to service delivery VPQM, Scottsdale AZ, january 2010

6 6 Subjective tests to determine the QoE was run over 6 weeks Tests were run in-service on VG LIVE, a live streaming service of Norwegian football 18 users: 14 male / 4 female, years old Similar to Single Stimulus Continuous Quality Evaluation Feedback through survey VPQM, Scottsdale AZ, january 2010

7 7 The user was asked to rate the VG LIVE service related to various aspects of the QoE Overall quality Adapting period Instant start/no buffering Video and audio flow Coding artifacts: Blurring Edge ringing Color bleeding Motion blurring VPQM, Scottsdale AZ, january 2010

8 8 Key finding 1: the QoE of Internet television is better than that of Internet video, but worse than that of broadcast TV VPQM, Scottsdale AZ, january 2010

9 9 Key finding 2: the results indicate that content, as well as context, can influence the QoE VPQM, Scottsdale AZ, january 2010

10 10 Key finding 3: service providers can receive feedback on the QoE in 4 stages 1. Full reference objective quality assurance at headend 2. Monitoring the network using bit rate parameters from the client 3. Extended user profile to include content/context information 4. QoE tool in the client for real time feedback, support and customization VPQM, Scottsdale AZ, january 2010

11 11 Summary A classification of QoE artifacts related to the service delivery of Internet television. Subjective tests show there is a need for improving the QoE of Internet television. A 4-stage QoE assessment plan from headend to user end can give the service provider a good measurment of the QoE for each individual user. Psychological factors such as content and context will need to be further researched in order to assess its impact on the QoE. VPQM, Scottsdale AZ, january 2010

12 12 Questions? VPQM, Scottsdale AZ, january 2010

13 13 Appendix

14 14 Internet television differs from IPTV in network openness, and from Internet video in production quality and business model

15 15 The service delivery of VG LIVE using streaming technology from Move Networks

16 16 The proposed QoE menu with 3 main features: QoE guide (1), support and user feedback (2), options and personalization (3).

17 17 Demographical/psychographical data 1

18 18 Demographical/psychographical data 2

19 19 User equipment 1

20 20 User equipment 2

21 21 Quality analysis 1: Quality comparison

22 22 Quality analysis 2: Details and sharpness

23 23 Quality analysis 3: Adaptation time

24 24 Quality analysis 4: Response time

25 25 Quality analysis 5: Audio/video smootheness

26 26 Quality analysis 6: Audio/video freeze/blackout

27 27 Post-survey 1: Overall impression

28 28 Post-survey 2: Function rating

29 29 Post-survey 3: Adaptive streaming and quality variations

30 30 Post-survey 4: Value of the service