Is it 4K? Is it 4k? UHD-1 is 3840 x 2160 UHD-2 is 7680 x 4320 and is sometimes called 8k

Transcription

1 So what is UHDTV? Ultra High Definition TV Richard Salmon - BBC 25 February 2015 RWTH Aachen University Trends in Video Analysis, Representation and Delivery Is it 4K? Is it 4k? UHD-1 is 3840 x 2160 UHD-2 is 7680 x 4320 and is sometimes called 8k 2 What resolution can the eye see? The historically accepted figure is: One minute of arc But how does this relate to the resolution of a TV system? BBC Research & Development White Paper WHP 092 Tests of visual acuity to determine the resolution required of a television transmission system John Drewery and Richard Salmon September Horizontal sampling of TV standard required for different screen sizes, viewed at 2.7m BBC Research & Development White Paper WHP There is a general consensus of opinion in the display industry that the preferred size for large domestic TV screens is likely to settle at between 37 and 42 inches as the flat-screen revolution matures

2 Do we need UHDTV Resolution? BBC White Paper 287 A Survey of UK Television Viewing Conditions Katy Noland and Louise Truong January Do we need UHDTV Resolution? Results from 2185 respondents in UK Median 16:9 diagonal screen size is 39.3 In 2004 we predicted 37 to Do we need UHDTV Resolution? Results from 2185 respondents in UK Median 16:9 diagonal screen size is 39.3 Median viewing distance is 2.63m Our previous study (102 respondents in 2004) reported a figure of 2.7m From which we conclude living rooms have remained much the same size! Do we need UHDTV Resolution? Results from 2185 respondents in UK Median 16:9 diagonal screen size is 39.3 Median viewing distance is 2.63m Median relative viewing distance is 5.5H HDTV satisfies viewers sitting at distances above 3H 9 10 Do we need UHDTV Resolution? Results from 2185 respondents in UK Median 16:9 diagonal screen size is 39.3 Median viewing distance is 2.63m Median relative viewing distance is 5.5H 10.2% sitting at 3H or closer 1.5% at 1.5H or closer Do we need UHDTV Resolution? Survey asked respondents what screen size they would ultimately prefer in the future Median ideal screen size: 48.2 inches If viewing distances don t change: Median relative viewing distance for ideal screen is 4.5H 22.9% sitting at 3H or closer 4.8% at 1.5H or closer

3 Do we need UHDTV Resolution? 10.2% would benefit now (but only if they had a 4k TV) 22.9% will benefit in the future Sales of 4k TV sets, currently usually 50 diagonal, are increasing rapidly: Ø 200,000 in 2014 Ø 995,000 in 2015 (GFK projection) So what is UHDTV? UHD-1 is 3840 x 2160 UHD-2 is 7680 x 4320 we have UHD-1 Phase 1 and UDH-1 Phase 2 and narrowly missed having Phase 0! UHD-2 NHK Super Hi-vision inch Viewing distance: 1-2m BBC and NHK SHV Collaboration Live London-Amsterdam demonstration (September 2008) inch Viewing distance: 65cm Viewing distance : 0.75 x Picture height Viewing angle : 100 degrees A3 wide Viewing distance: 25cm BBC and NHK SHV Collaboration Live London-Tokyo tests (September 2010) Proving test for 2012 Olympics Live delivery of SHV half-way round the world Over IP networks Two official tests: Charlatans Radio 6 Music session (Day 1) Taekwondo demonstrations (Day 2) BBC and NHK SHV Collaboration

4 25/02/2015 BBC and NHK SHV Collaboration BBC and NHK SHV Collaboration Local viewing on a 4k 103in display Charlatans 19 BBC and NHK SHV Collaboration 20 BBC and NHK SHV Collaboration Taekwondo BBC and NHK at London 2012 Olympics Live and pre-prepared demos over Internet presented on large screens in theatres in Glasgow, Bradford, London, New York and three locations in Japan 21 BBC and NHK SHV Collaboration 22 UHD-2 NHK Super Hi-vision NHK will launch UHD-2 service with Tokyo 2020 Olympics with a trial service in

5 UHD cameras - Large formats The first experimental HDTV cameras used large format sensors and now we find ourselves in the same situation for UHDTV UHD cameras - Large formats Sony F65 RED Dragon Sony F UHD cameras - Large formats Shallow depth of field Intended for digital-cinema and not TV Sports coverage requires greater depth of field Higher frame rates Panasonic 2/3-inch Varicam HS Shown at NAB HD sensors up to 240 fps Panasonic Varicam 35 What exactly is UHD-1 ITU-R BT.2020 Single 35mm 4k sensor up to 120 fps 3840 x 2160 Wider colour gamut Constant Luminance option Greater bit depth Higher frame rates Improved Audio

6 Wider Colour Gamut Wider Colour Gamut HDTV (Rec 709) UHDTV (Rec 2020) DCI P3 Pointer s gamut of real surface colours HDTV (Rec 709) UHDTV (Rec 2020) Not everyone convinced we should change SMPTE retains 709 gamut as option for UHD Proposals to move to XYZ like digital cinema Constant Luminance Constant Luminance Conventional TV (and computer imagery) takes R, G and B, and applies gamma precorrection (R, G, B ). Luma (Y ) and Chroma (C B, C R ) is matrixed from R, G, B Bandwidth limit (sub-sample) the Chroma Conventional TV thus results in: A proportion of what the eye perceives as Luminance travelling through the Chroma channel, with resultant loss of resolution Some high-frequency Chrominance information which they eye cannot see consumes data capacity in the Luma channel Constant Luminance Constant Luminance CL attempts to rectify this by forming True Luminance and Chrominance signals from R, G, B before gamma pre-correction CL attempts to rectify this by forming True Luminance and Chrominance signals from R, G, B before gamma pre-correction Available as an option in Rec.2020 Resistance to its adoption

7 Greater bit depth HDTV is produced at 10-bits per component and transmitted at 8-bits per component UHDTV should be produced at 12-bits per component and transmitted at 10-bits. This gives improved compression performance Note that this is bit-depth in the gamma-corrected (nonlinear) world, not linear light The need for a Higher Frame Rate Even at HD resolutions, motion blur due to camera integration reduces the resolution of the image The need for a Higher Frame Rate The need for a Higher Frame Rate When the camera pans the entire High Definition scene becomes blurred - for example when following the action during a football match As you increase the resolution so the rate of panning has to be reduced to keep the blurring under control The need for a Higher Frame Rate The dynamic resolution of HDTV is no better than SD Impact on the Viewer Where there is a large difference between the resolution of a static and dynamic picture, this can lead to a feeling of nausea Therefore the higher the static resolution, the higher the dynamic resolution must be for comfortable & lifelike images

8 Up-converting Displays TVs claiming 100/120Hz, 200/240Hz or even 600Hz frame rates are now common These put 50/60 fps signals on the screen interpolated to higher rates, so solving the problems of flicker and display smearing they create intermediate pictures using motion prediction, or insert black fields it is done to mitigate the problems of sampleand-hold displays Up-converting Displays But this is not High Frame Rate TV Cannot reduce motion blur captured in camera Cannot predict complex motion To make motion rendition more lifelike we need higher frame rates in the camera for distribution and in the display Higher Frame Rates I would suggest that: if SD is acceptable at 50 fps then full HDTV needs around fps as resolution increases, do we need 300 fps? Probably not! Ways to achieve Higher Frame Rates We need to consider the whole system to see how higher frame rates could be accommodated 1. Flicker in the display 2. Length of Shutter Opening for sharp images 3. Ability of HVS to fuse motion 4. Lighting issues 5. Noise performance : Length of Shutter Opening for sharp images Experiments at NHK conducted to determine the frame rate for a future UHDTV system: Require > 80 fps to prevent flicker on a large screen at normal TV brightness Require shutter opening < 1/320 second to prevent blur Sony tested sequences between 60 & 480 fps Require ~250 fps to prevent blur 3: Ability of HVS to fuse motion Need to consider: i. Trackable motion ii. Non-trackable motion iii. Multiple trackable motions, of which the eye will only track one

9 3: Ability of HVS to fuse motion Trackable motion (linear motion, where the eye can join the dots caused by a short shutter opening) Tests indicate that fps may be sufficient 3: Ability of HVS to fuse motion Trackable motion (linear motion, where the eye can join the dots caused by a short shutter opening) Non-trackable motion rotation : Ability of HVS to fuse motion 3: Ability of HVS to fuse motion Trackable motion, but which the eye is not tracking : Ability of HVS to fuse motion Experiments at NHK conducted to determine the frame rate for a future UHDTV system: Require > 80 fps to prevent flicker on a large screen at normal TV brightness Require shutter opening < 1/320 second to prevent blur Require >100 fps to merge motion in eye Hence current standards of 100 and 120 frames per second for UHDTV 3: Ability of HVS to fuse motion Sony tested sequences between 60 & 480 fps Require ~250 fps to prevent blur Require ~250 fps to prevent jerkiness Suggestion of 240 fps for compatibility with 24 and 60 fps

10 3: Ability of HVS to fuse motion What is the relative importance of: trackable motion (>100 fps adequate) and untrackable motion (1,400 fps required?) Latest work indicates up to 700 fps may be needed to completely match the eye s abilities so it comes down to what is feasible 4: Lighting Issues Could we cope with 120 fps production in Europe? Tungsten Fluorescent tubes High frequency HMI LED Study concluded we need 100 fps Intention to use 100 fps in Europe included in 2014 version of ITU : Noise performance Shorter exposure in camera = more noise How does Human Visual System respond to noise with increasing frame rate? Is the relationship linear? Is there a penalty? or could there be an advantage? 5: Noise performance How does Human Visual System respond to noise with increasing frame rate? Noise becomes less visible at higher frame rates Ultimately you can use a 1-bit system, as in DLP projector or plasma display c.f. 1-bit audio Experiment required to determine characteristic, but not highest priority Improved Audio NHK Super Hi-Vision 22.2 channel surround Improved Audio BBC Work Object-based audio Accessible experiences Immersive experiences Personalised experiences Interactive experiences

11 25/02/2015 Improved Audio BBC Work Traditional Broadcasting What is Object-Based Broadcasting? 61 Object-Based Broadcasting (OBB) 62 Object-Based Broadcasting (OBB) Content production for OBB Create content objects formed of individual pieces of audio or video Enables meaningful combinations in multiple ways Define relationships between them Define the overall user experience Examples of Object-Based Broadcasting Examples of Object-Based Broadcasting User experiences Accessible User experiences - Immersive Key driver for OBB Future proof representation Enables AV content to be tailored to an individual's needs e.g. providing variable sound mixes Can be rendered in various ways based on a single representation

12 25/02/2015 Examples of Object-Based Broadcasting Examples of Object-Based Broadcasting User experiences - Binaural Audio User experiences - Interactive Enables new headphone experience Can create interactive experiences for audiences - Binaural postprocessing Gives audiences greater control over the video, audio and on-screengraphics - or dedicated mixing for headphones Examples of Object-Based Broadcasting Object-Based Broadcasting User experiences Responsive Enabler for new user experiences spanning the range from traditional broadcasting to interactive experiences Technology and standards are emerging Tools need to be made available to enable and explore a wider range of applications Curation of content is more important then ever to enable these new experiences and ensure satisfactory results Object-Based Broadcasting has the potential to be transformational Enables dialog enhancement Adjust Foreground/ Background balance Pause function with background music Adjust the length of a programme without producing different versions Not yet standardised Subject of a Rapporteur s Group meeting at ITU last week Variety of proposals put forward An ounce of contrast is worth a whole heap of resolution

13 Variety of proposals put forward USA (Dolby / based on SMPTE ST-2084) Philips Yuv proposal BBC Gamma plus Log proposal NHK Shifted Gamma proposal Dolby / SMPTE ST-2084 Perceptual coding of absolute brightness Peak brightness of 4,000 or even 10,000 cd/m 2 (nits) Perceptual quantiser not ideal for relative brightness Perceptual quantiser may have to be un-done for image scaling, resizing and filtering Dolby / SMPTE ST-2084 Perceptual coding of absolute brightness Black level extended for controlled cinema environment Our survey of domestic viewing environment shows that medium or bright room lighting is preferred by 59% of viewers Dolby / SMPTE ST-2084 Perceptual coding of absolute brightness Black level extended for controlled cinema environment Optimised for end-to-end metadata describing the "mastering" display Not a practical solution for live TV / presentation / mixed with graphics Philips Yuv proposal Very pure colorimetric approach Works best for a Constant Luminance system Not backwardly compatible for TV BBC Gamma plus Log proposal NHK Shifted Gamma proposal Both aim for a measure of backward compatibility Both very suitable for TV Both aim for very low cost of implementation Avoid metadata Restrict brightness range so as to avoid viewer adaption

14 Last week: Dual approach agreed Two parallel approaches to be developed: USA (Dolby / based on SMPTE ST-2084) Philips Yuv proposal BBC & NHK joint proposal SMPTE ST-2084 This standard specifies an EOTF characterizing high-dynamic-range reference displays used primarily for mastering non-broadcast content. This standard assumes a peak luminance level limited to 10,000 cd/m 2. Cinema: 48 cd/m 2, TV Production: 100 cd/m 2, Domestic TV: cd/m SMPTE ST-2084 SMPTE ST-2084 The linear color values proportional to the desired optical output are related to the nonlinear color values proportional to an input. This relationship shall be defined by the EOTF linearization equation, constructed to align with human visual contrast sensitivities over a specific range of luminance values, therefore the absolute optical output shall be defined In other words, a code value in the video corresponds to an absolute luminance level produced from the monitor. Reference viewing environment for Digital Cinema Black, non-reflective finishes on all surfaces, recessed lighting which is appropriately masked and filtered, and an ambient light level reflected by the screen of less than 0.01 cd/m 2. Reference viewing environment for HDTV Room illumination of 10 Lux, chromaticity of background as D 65, and a luminance of background of 8 to 12 cd/m BBC Higher Dynamic Range approach Based on Rec.709/2020 OETF for compatibility Simply increasing bit depth is not efficient With ten bits our HDR proposal can support dynamic range about 100 fold greater than SDR television. Based on relative brightness, in that same way as conventional television, which is both simple and, we believe, the correct approach for HDR TV. BBC Higher Dynamic Range approach Rec.709/2020 has a linear section near black to limit noise in blacks Modern cameras have much improved sensitivity and noise performance, so this limit on dynamic range is no longer desirable so BBC proposal removes this constraint

15 BBC Higher Dynamic Range approach Follows Rec.709/2020 exponential curve up to 50% signal level Blends gently into a log curve to handle highlights Similar in concept to a conventional camera with knee function, but well defined Well matched to human visual system Display screen brightness For TV the eye should not be required to adapt For long-term viewing, images should be comfortable to view There is a widespread view that 1,500 to 2,000 cd/m 2 is sufficient for domestic (indoor) TV viewing Display screen brightness Concern over viewing bright images late at night melatonin Concern in EU over power consumption What exactly is UHD-1 ITU-R BT x 2160 Wider colour gamut Constant Luminance option X Greater bit depth Definitely 10-bit for Tx Higher frame rates 100/120 fps for some genres Improved Audio Object based, or hybrid Increased dynamic range beyond ITU BBC tests from UEFA World Cup 2014 Recent BBC Experimental Transmissions 3840x2160p at 60 fps Live delivery by DVB-T2 and MPEG-DASH

16 25/02/2015 BBC tests at Commonwealth Games 2014 BBC tests at Commonwealth Games x2160p at 50 fps Near-live presentation of High Frame Rate TV Live (distributed) production over IP Live delivery by DVB-T2 from 3 main transmitters Live delivery using MPEG-DASH over Internet 91 4k HFR camera, HD 100fps presentation 92 BBC 4k / UHD BBC 4k / UHD Small number of Productions in 4k 25p Small number of Productions in 4k 25p Drama, Natural History High-value productions Archive value Drama, Natural History 4k 25p used for Queen s Christmas Message Main output at HD Delivered over iplayer at 4k UHDTV Broadcasting When? UHD-1 Phase 1 possible now No EBU Members plan to use Phase 1 Will it be a Channel? Likely initially to be one-offs delivered over the Internet 95 Thank you richard.salmon2@bbc.co.uk 96 16