BT Media & Broadcast. Research Paper

Transcription

1 BT Media & Broadcast Research Paper

2 BT Media and Broadcast Research Paper Ultra High Definition Video Formats and Standardisation Version 1.0 April 2015 Mike Nilsson Page 1

3 Contents Page 1 Introduction 6 2 Enhanced Resolution 7 3 High Dynamic Range The Benefit of High Dynamic Range The dynamic range of the human visual system The non-linearity of the human visual system The mapping of linear light to code levels The mapping of pixel code levels back to linear light Black level: how dark should displays be? The mapping of pixel code levels to linear light in the presence of ambient light The current state of high dynamic range capture and display technology Interest and Experience in Hollywood Dolby Cinema 22 4 Wider Colour Gamut The CIE RGB Colour Space The CIE XYZ Colour Space Perceptually Uniform Colour Spaces Colour Television The need for a Wider Colour Gamut Wider Colour Gamut Standards Conversion between Colour Gamuts How large a Colour Gamut is needed? Display Technology for Wider Colour Gamut 31 5 Higher Frame Rate The artistic impact of frame rate Historical choices of frame rate The motion blur jerkiness trade-off Subjective evaluation of moving picture quality 35 Page 2

4 5.5 The impact of lighting frequency on frame rate 38 6 Standardisation ITU-R Parameters for Digital Television High Dynamic Range Colorimetry conversion Higher Frame Rates SMPTE UHD Parameter Values High Dynamic Range Colour Volume Metadata Digital Cinema Colour Equations On-going Activities MPEG High Efficiency Video Coding The Future of Video Coding Standardisation High Dynamic Range and Wide Colour Gamut DVB UHD-1 Phase UHD-1 Phase UHD Eco-design requirements for electronic displays EBU DTG Blu-ray Disk Association HDMI Forum The Digital Cinema Initiatives (DCI) The Forum for Advanced Media in Europe (FAME) UHD Alliance and UHD Forum 54 7 Abbreviations 55 Page 3

5 BT Media and Broadcast contacts Name Business Area / Role Phone John Ellerton Head of Media Futures john.ellerton@bt.com Jonathan Wing Head of Sales jonathan.wing@bt.com BT Research and Innovation contacts Name Business Area / Role Phone Mike Nilsson Research and Innovation mike.nilsson@bt.com Steve Appleby Research and Innovation steve.appleby@bt.com Page 4

6 Executive summary BT Media and Broadcast provides services to broadcast and media organisations worldwide, including the international carriage of Ultra High Definition Television signals. We commissioned our colleagues in BT Research and Innovation to examine the current international status of technology and standardisation of Ultra High Definition Television to aid in our own understanding and future product development. This paper is the result, which we were so impressed with, we decided to release as a resource to the professional broadcast community. We hope it is useful do get in touch if you have feedback or comments. UHD televisions are now retailing in significant numbers, and services, such as those offered by Netflix and Amazon, are starting to appear in the market. But while these services offer higher resolution than HD services, further improvement could be made in due course to provide an even better viewing experience. Future television systems should be capable of producing an experience that is either closer to real life or is capable of more accurately recreating the artistic intent of the storyteller. To this end, increased resolution, wider colour palette, higher frame rate and an improvement in the dynamic range of the images when used together, have the potential to provide viewers with a better visual experience compared to current television applications and provide a viewer with a stronger sense of being there. Many standardisation organisations around the world have been, and are continuing to be, very active in the area of UHD TV standardisation. DVB has created standards for what it terms UHD-1 phase 1, effectively the parameters of HDTV but with enhanced resolution, similar to the deployments of Netflix and Amazon. DVB is now working on the commercial requirements for two subsequent phases, the first of which is expected to add support for higher dynamic range, wider colour gamut, and higher frame rates, and the second of which is planned to add support for even higher, 8K, resolution. The HEVC compression standard, almost essential to make delivery of UHDTV commercially feasible, has been approved, with the recently approved second version including support for higher bit-depths and enhanced chroma formats. MPEG is currently studying whether HEVC is optimal, as currently standardised, to support higher dynamic range and wider colour gamut, and if not, is expected to launch a standardisation activity during 2015 to address the deficiencies. There is widespread interest in backwards compatibility of future UHDTV services, which may have higher dynamic range, wider colour gamut, and higher frame rates, with first generation UHDTV services with only enhanced resolution. Technology to achieve this is still under development, and the ultimate success or failure of backwards compatibility will depend on how efficiently it could be implemented compared to simple but inefficient simulcasting. There are, at the writing, many problems and few agreed solutions, but there are many lively discussions taking place in the standards community could be the year in which significant advances are made to the technical standardisation of the second phase of UHDTV. This report begins with sections describing the science and technology of UHD television: enhanced resolution, higher dynamic range, wider colour gamut and higher frame rate. This is followed by a section describing the current state of standardisation in various bodies that are essential to the standardisation of UHD television services, ITU-R, SMPTE, MPEG and DVB, while also providing a brief status update on EBU, DTG, Blu-ray Disk Association, HDMI Forum, Digital Cinema Initiatives (DCI), the Forum for Advanced Media in Europe (FAME), the UHD Alliance and the UHD Forum. We consider that while the enhanced resolution of the first phase of deployments of UHDTV services could provide undoubtedly better picture quality than current HDTV services when viewed from an optimal viewing distance, the combination of human visual acuity, screen size, and home viewing distances could make the improvement over HDTV in the home environment less pronounced. We have observed the growing feeling within the standardisation community that to make UHDTV reach the mass market, additional features would be needed beyond the enhanced resolution achieved with the first phase of deployment. In particular, enhancements that that do not depend so critically on viewing distance would be beneficial. Fortunately these features, higher dynamic range, wider colour gamut, and higher frame rates, are becoming technologically feasible and are generating much interest within the industry. This has perhaps been most noticeable in the film industry. Dolby Cinema, which was announced in December 2014, and which is claimed to deliver high dynamic range with enhanced colour, is planned for theatrical exhibition in The Blu-ray Disc Association is developing a next generation format that will support UHD, a wide colour gamut, and high dynamic range, with the first players and titles being expected before the end of The broadcast TV industry is close behind, with DVB aiming to develop specifications to enable deployment of second phase UHD services in the timeframe. Page 5

7 1 Introduction Ultra High Definition TV has attracted a good deal of attention lately as a result of a push from TV vendors and some content providers, notably Netflix. However, to date, the focus has been solely on increased resolution. Ultra High Definition though, is about more than just increasing the number of pixels, being also about increasing the dynamic range, widening the colour gamut and increasing the frame rate. When viewing TV on a popular size of screen in a typical home environment, it is these additional improvements which may ultimately provide the most benefit to the viewer. The quality of television is therefore set to improve in multiple dimensions over the next few years. There is still heated debate about the standards that will be used, but it seems clear that managing legacy and dealing with multiple dimensions of TV improvements will be a challenge for the broadcast industry. The purpose of this document is to provide a thorough review of the current status of UHD standards with a view to obtaining a clearer understanding of the challenges and opportunities ahead. Page 6

8 2 Enhanced Resolution Display and capture technology is advancing. Today there are both video cameras and displays capable of resolutions higher than full High Definition TV. These resolutions are frequently referred to as either 4K or Ultra High Definition TV (UHDTV) and have resolution 3840x2160, that is, four times the resolution of full High Definition TV. Figure 1 shows the resolution of UHD relative to the earlier formats of Standard Definition (SD), HD-ready (720p HD), and High Definition (HD). Figure 1. The resolution of UHD relative to earlier, lower, resolutions. UHD televisions are now retailing in significant numbers 1, enabling potentially much better picture quality than current HD televisions. BT worked with the BBC during 2014 to deliver coverage of the 2014 football World Cup Final in UHD resolution live to the BT Tower, using HEVC video compression and MPEG DASH technology, delivering content from BT Wholesale Content Connect over BT Infinity superfast fibre optic broadband through BT Home Hub 5 routers to three set top boxes connected to UHDTVs and directly to two internet-enabled UHDTVs. The picture quality achieved was very impressive. The quality of the UHD signal was undoubtedly much better than HD delivered by Freeview, when viewed near to the screen. The ITU-R describe optimal viewing distance in image heights (H) for various digital image systems 2, recommending 6H for Standard Definition, 4.8H for 720p HD, 3.2H for HD, and 1.6H for UHD. Such optimal viewing distances are often calculated by considering normal human visual acuity to be 20/20, meaning that a letter, such as the letter E, can be identified when top to bottom it subtends an angle of 5 arc minutes. This corresponds to 1 arc minute per horizontal feature (three lines and two spaces). It also corresponds to one dark to light cycle every 2 arc minutes. One dark to light cycle every 2 arc minutes equals 30 cycles per degree, which can be represented with 60 pixels per degree. This has been taken as standard acuity for TV applications, although clearly some viewers have worse eye sight and some have better eye sight. The screen diagonal size, S, measured in inches, required to achieve a resolvable number of horizontal pixels, R, when viewed from a distance D, measured in centimetres, is given by the following equation, assuming visual acuity of 60 pixels per degree and 16:9 picture aspect ratio. R π S = tan ( ) "Guidelines on metrics to be used when tailoring television programmes to broadcasting applications at various image quality levels, display sizes and aspect ratios", Recommendation ITU-R BT (03/2010), I Page 7

9 However, there is reason to believe that viewers will not sit as close as 1.6H to their television, and hence would not achieve the ITU-R optimal viewing experience. A survey 3 of 102 BBC employees in 2004 reported that the median viewing distance for viewing the main TV in the home was 2.7m. The BBC carried out a more extensive survey of the UK population during the summer of , collecting information from 2185 people who have a television in their home, and finding the median viewing distance was 2.63m. While the 2004 survey found a median screen height of 32.5cm, the median height from the 2014 survey was 49cm, which, assuming a 16:9 picture aspect ratio, corresponds to a screen diagonal size of 39.3 inches. The 2014 survey also found that the median relative viewing distance is 5.5 times the screen height (5.5H). The 2014 survey, by asking respondents how they would expect the size of their next television to compare to that of their current one, found that about half the respondents expected to buy a larger screen next time. When asked to estimate the ideal television size for their current home, assuming that money were no object, they responded with a median ideal diagonal size of 48 inches. By assuming that people would need to sit three picture heights (3H) or closer to a UHD screen to get a resolution improvement over HD, the BBC deduced from the 2014 survey that 10.2% of viewers would be able to observe a benefit from UHD with their current television, and 22.9% would if they acquired their ideal television. These two surveys show no significant change in television viewing distances over a period of 10 years. As this is likely to be significantly influenced by the size of rooms in UK homes, it would seem reasonable to expect little change in the coming years. Currently 55 inch and 65 inch diagonals are popular for UHD televisions. There would appear little reason to believe that this would change in the near future, as it is in part determined by the size and layout of rooms in which people watch television. A viewing distance of 1.6H, which the ITU-R consider to be optimal for viewing UHD content, when calculated for a television with 65 inch diagonal, corresponds to a viewing distance of about 1.3m. It seems unlikely given the results of the two BBC surveys that many people would sit this close to their UHDTV. Table 1 shows the resolution of various popular TV formats and the screen size that would be needed for optimal viewing at a distance of 2.63m, the median viewing distance found in the 2014 BBC survey. It is unlikely that viewers would want to, and be able to, acquire screens with diagonal size of 148 inches, so that they could both maintain a viewing distance of 2.63m and achieve optimal, as defined by the ITU-R, viewing of UHD content. While some viewers may compromise on viewing distance and screen size to get such a UHD experience, it seems likely that the majority would not, and hence they may not get the full benefit of UHDTV resolution. Format Horizontal Resolution Screen Size (Inches) SD p HD HD UHD Table 1. The screen size needed for optimal viewing of popular TV formats from 2.63m. The screen would not need to be so large if the requirement were relaxed from achieving optimal viewing of UHD content to achieving a better experience than watching HD content. Again referring to Table 1, any screen with diagonal size larger than 68 inches would enable more resolution than HD to be observed from 2.63m. 3 Results of a survey on television viewing distance, N. E. Tanton, June A Survey of UK Television Viewing Conditions, Katy C. Noland and Louise H. Truong, January 2015 Page 8

10 However, there is some disagreement with this whole methodology. Spencer 5 has reported results of subjective tests in which viewers assessed content from a distance of 30cm, controlled using a head mount. By using the methodology described above, the viewers should have been limited to seeing benefits as the pixel density increased to about 300 pixels per inch, and then seeing no further improvement as the pixel density was increased. However, the presented results show improvements at 500 pixels per inch, and yet further improvements at 1000 pixels per inch. Spencer explains these results by stating that the human eye is able to detect differences represented by even finer structures, referencing tests of vernier acuity, which involve distinguishing the relative alignment of parallel lines, that show that humans can distinguish details five to ten times smaller than standard resolution measurements predict. While this is an isolated study, it does suggest that further investigation of the benefits of the increased resolution of UHDTV, when viewed on televisions of desirable size at normal home viewing distances, would be worthwhile. 5 How much higher can mobile display resolution go?, Lee Spencer, Page 9

11 3 High Dynamic Range The human visual system has a significantly larger dynamic range than that supported by current television systems. Display devices that are able to support a much larger dynamic range than conventional televisions systems are starting to appear in the market. They achieve the larger dynamic range by extending the range both towards darker as well as lighter values as a result of fitting a conventional LCD panel with a spatially varying backlight, which could be either a projector or a panel of LEDs which are individually addressable. A key technical challenge for video delivery is how to signal the higher dynamic range while also supporting legacy standard dynamic range displays. Some proprietary technology is already entering the market, and various standardisation bodies, including SMPTE, the EBU and MPEG are considering the issues. This section provides an overview of the benefits of using a higher dynamic range for images and video, the capability of the human visual system, and the capability of emerging display technology. Additionally, the issues and proposed solutions for integrating higher dynamic range into existing end to end television systems are discussed. 3.1 The Benefit of High Dynamic Range Figure 2 is an example of an image where the use of a higher dynamic range would improve the viewing experience. With a standard dynamic range, it is not possible to have visible details in both the shaded parts of the image and the parts that are in full sunshine. Figure 2. An image that would benefit from high dynamic range. The use of a high dynamic range is not simply making images brighter, which in itself can be effective, but enabling detail to be seen in both dark and light areas. Figure 3 shows an example of a histogram of the pixel luminance values of an image, represented with both standard and high dynamic range. Many of the pixels have about the same luminance in both cases, and only some pixels, the lowlights and the highlights in the image, have lower or greater luminance in the high dynamic range variant. Luminance is measured in the derived SI units of candela per square metre (cd/m 2 ), also known as the nit (1 nit = 1 cd/m 2 ). Typically displays are limited to a dynamic range of 100:1, and a typical maximum luminance of 100 cd/m 2 due to legacy limitations of CRT display technology and the derived current specification for display interfaces. The human visual system, as described later, is capable of perceiving a much higher dynamic range and higher levels of luminance. Figure 4 illustrates the wide range of luminance values that occur in real world scenes, labelling specific features with approximate values of light level. Page 10

12 Relative Frequency Luminance (cd/m2) Standard Dynamic Range Image High Dynamic Range Image Figure 3. An example pixel luminance histogram for a standard and high dynamic range image. 4,000 nits 10,000 nits 50,000 nits 40 nits 100 nits 2 nits 3.2 The dynamic range of the human visual system Figure 4. Examples of approximate light levels in real world scenes. The human visual system has a huge dynamic range, being able to cope with starlight at 10-4 cd/m 2 and bright sunshine at 10 5 cd/m 2. Hood and Finkelstein 6 report values ranging from 10-6 to 10 cd/m 2 for scotopic light levels where light transduction is mediated by the rod cells in the eye, and from 0.01 to 10 8 cd/m 2 for the photopic range where the cone cells are active. And in the overlap, called the mesopic range, both rods and cones are involved. At any one time, the human visual system is able to operate over only a fraction of this enormous range. This subset is called the simultaneous or steady-state dynamic range. It shifts to an appropriate light sensitivity due to various mechanical, photochemical and neuronal adaptive processes 7, so that under any lighting conditions the effectiveness of human vision is maximised. 6 From D. C. Hood and M. A. Finkelstein, Visual sensitivity, in Handbook of Perception and Human Performance, K. Boff, L. Kaufman, and J. Thomas, Eds., vol. 1. Wiley, New York, J. A. Ferwerda, Elements of early vision for computer graphics, IEEE Computer Graphics and Applications 21, 5, 22-33, luthuli.cs.uiuc.edu/~daf/courses/rendering/papers3/ pdf. Page 11

13 The simultaneous dynamic range over which the human visual system is able to function can be defined as the ratio between the highest and lowest luminance values at which objects can be detected, while being in a state of full adaptation. In a given room that contains a display device, the human visual system will typically be in a steady state of full adaptation. Figure 5, taken from Hood and Finkelstein, shows the overall range of the human visual system compared to the ranges of its approximate steady state as well as the range for typical Low Dynamic Range (LDR) and High Dynamic Range (HDR) displays. While the overall dynamic range of the human visual system and these display devices is known, less is known about the range of the human visual system in the steady-state. Kunkel and Reinhard 8 have reported a sequence of psychophysical experiments, carried out with the aid of a high dynamic range display device, where they determined the simultaneous dynamic range of the human visual system, finding the human visual system to be capable of distinguishing contrasts over a range of 3.7 log units, equivalent to a range of about 1:5000, under specific viewing conditions. They also found that the dynamic range is affected by stimulus duration, the contrast of the stimulus and the background illumination, which they claimed accounts for the different dynamic ranges being reported in the literature. Figure 5. The dynamic range of the human visual system. Jenny Read, neuroscientist at the University of Newcastle, talked at DVB-EBU HDR Workshop in June on how the Human Visual System reacts to high dynamic range screens, stating that encoding schemes must be defined by taking into account human contrast sensitivity. It is also necessary to take into account the human visual system adaptation states, where light to dark adaptation is slow, typically taking between 8 and 30 minutes, and where dark to light adaptation is much quicker, typically taking just a few minutes. Daly et al 10 report studies to find the dynamic range that is preferred by human observers. They used a Dual Modulation Research Display, as shown in Figure 6, and referenced by Hammer 11, which is capable of supporting a dynamic range from cd/m 2 to 20,000 cd/m 2, that is, a contrast ratio of 5,000,000:1; and which supported the DCI P3 colour gamut. The experiment used several realistic and synthetic stimuli in a dark viewing environment. They found for diffuse reflective images a dynamic range between 0.1 and 650 cd/m 2 matched the average preferences, but to satisfy 90% of the population, a dynamic range from to about 3000 cd/m 2 is needed. They claim that since a display should be able to produce values brighter than the diffuse white maximum, as in specular highlights and emissive sources, they conclude that the average preferred maximum luminance for highlight reproduction satisfying 50% of viewers is about 2500 cd/m 2, increasing to marginally over cd/m 2 to satisfy 90%. They conclude that there would be a benefit from more capable displays, as the preferred luminances found in their study exceed even the best of consumer displays today. 8 T. Kunkel and E. Reinhard, A reassessment of the simultaneous dynamic range of the human visual system, Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, ISBN , pp , July DVB-EBU HDR Workshop, IRT, Munich, Germany, 17 June 2014, 10 S. Daly, T. Kunkel, S. Farrell & Xing Sun: Viewer Preferences for Shadow, Diffuse, Specular, and Emissive Luminance Limits of High Dynamic Range Displays. SID Display Week Hammer, High-Dynamic-Range Displays, Page 12

14 Figure 6. The experimental set up used by Daly et al. to assess preferred dynamic range. Hanhart, Korshunov, and Ebrahimi 12 report a set of subjective experiments to investigate the added value of higher dynamic range. Seven test video sequences at four different peak luminance levels were assessed using the full paired comparison methodology. Pairs of the same sequence with different peak luminance levels were displayed side-by-side on a Dolby Research HDR RGB backlight dual modulation display (aka Pulsar ), which is capable of reliably displaying video content at 4000 cd/m 2 peak luminance. Their results, as shown in Figure 7, show that the preference of an average viewer increases logarithmically with the increase in the maximum luminance level at which the content is displayed, with 4000 cd/m 2 being the most attractive option of those tested. Figure 7. Results of subjective evaluation of HDR Video using pair comparison by Hanhart, Korshunov, and Ebrahimi. 3.3 The non-linearity of the human visual system Ernst Heinrich Weber studied of the human response to a physical stimulus in a quantitative fashion, finding, in what is now known as Weber's law, that the just-noticeable difference between two stimuli is proportional to the magnitude of the stimuli, that is, an increment is judged relative to the previous amount. Gustav Theodor Fechner used Weber's findings to construct a psychophysical scale in which he described the relationship between the physical magnitude of a stimulus and its (subjectively) perceived intensity. 12 Subjective Evaluation of Higher Dynamic Range Video, Philippe Hanhart, Pavel Korshunov, and Touradj Ebrahimi, SPIE Optical Engineering + Applications, Applications of Digital Image Processing XXXVII. Page 13

15 Fechner's law states that subjective sensation is proportional to the logarithm of the stimulus intensity. Fechner scaling has been found to apply to the human perception of brightness, at moderate and high brightness, with perceived brightness being proportional to the logarithm of the actual intensity 13. At lower levels of brightness, a more accurate description is given by the de Vries-Rose law which states that the perception of brightness is proportional to the square root of the actual intensity. 3.4 The mapping of linear light to code levels Current television systems typically support content with a range of brightness from about 0.1 cd/m 2 to about 100 cd/m 2. Consequently, it is the de Vries-Rose characteristic, described above, that has been designed into television systems to date, where a function known as gamma or gamma correction, is used to map brightness (linear light) to a scale in which each increment corresponds to a constant perceptual change. This function is known as the Opto-Electrical Transfer Function (OETF). In analogue television systems this characteristic made the whole range of intensities equally sensitive to noise, whereas in digital television systems it enabled the number of bits needed to represent each sample to be minimised 14. This characteristic is shown in Figure 8, where mappings, as specified in ITU-R Recommendation BT.709, to both 8 and 10 bit code levels are shown. The use of 10 bit codes, as increasingly used in video production, gives little benefit over 8 bits if BT.709 is still used, as the extra bits are the least significant bits, which are often discarded for display on 8 bit devices, and which at best could reduce the minimum black level, that is, make the blacks even blacker. While using BT.709, the extra two bits cannot increase the brightness, even though this would often be more desirable than increasing the maximum darkness. ITU-R Recommendation BT.2020, a more recent specification than BT.709, defines higher frame rates and a wider colour space, but the same OETF as BT.709. These and other ITU-R Recommendations are discussed in further later in this research paper. The dynamic range of a video signal can be increased by using an alternative Opto-Electrical Transfer Function (OETF) to that specified in BT.709. Digital Imaging and Communications in Medicine (DICOM) is a standard for handling, storing, printing, and transmitting information in medical imaging. This includes in Part an Opto-Electrical Transfer Function from linear light to 10 bit code levels, also shown in Figure 8, and valid between 0.05 and 4000 cd/m2 whereby each 1-bit luminance increment is equally visible according to Barten 16. Jenny Read stated at the DVB-EBU HDR Workshop in September 2014 that the film industry has used log-curves for many years that are quite near the human visual system and are much more efficient than the BT.709 Transfer Function used for TV today. A logarithmic OETF is used to map 14 stops of linear light, that is, a dynamic range of about 16,000:1, to a 10 bit signal, and is satisfactory to capture the Cineon format. Miller et al 17 of Dolby Laboratories have proposed a new OETF to provide higher dynamic range for video and movie production and distribution, which has recently been standardised by the Society of Motion Picture Engineers as SMPTE ST 2084:2014, High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays, which is described later in this research paper. This is also based on the work of Barten, and is also shown in Figure 8. But there is no industry consensus on the relevance of the work of Barten. The BBC in its white paper argues that Miller s proposal links the camera OETF to the absolute brightness of the display and that this has potentially far reaching consequences. 13 Weber Fechner law, 14 C. A. Poynton, Digital Video and HDTV: Algorithms and Interfaces, Electronics & Electrical, Morgan Kaufmann series in computer graphics and geometric modelling, ISBN , Barten, P.G.J., Physical model for the Contrast Sensitivity of the human eye. Proc. SPIE 1666, (1992); Barten, P.G.J., Spatiotemporal model for the Contrast Sensitivity of the human eye and its temporal aspects. Proc. SPIE (1993); and P. G. J. Barten, Formula for the contrast sensitivity of the human eye, Proc. SPIE-IS&T Vol. 5294: , Jan Scott Miller, Mahdi Nezamabadi and Scott Daly. Perceptual Signal Coding for More Efficient Usage of Bit Codes. SMPTE Motion Imaging Journal : doi: /j BBC White Paper WHP 283, Non-linear Opto-Electrical Transfer Functions for High Dynamic Range Television, T. Borer, July Page 14

16 Linear Luminance (cd/m2) The BBC argues that previously the photographic, movie and television industries have all always worked with relative, rather than absolute, luminance levels. The BBC argues that changing to absolute luminance levels will require significant changes to the way television is produced and viewed; and that it is not clear that the large dynamic range of 10 7 is needed for image display, when the simultaneous dynamic range of the eye is about They have developed their own proposal for an OETF, as shown in Figure 8, and which is imagined to be consistent with their submissions to ITU-R WP6C. There is on-going discussion of high dynamic range in standardisation bodies. The formal standardisation process for high dynamic range signal and exchange format, including the consideration of suitable Opto-Electrical Transfer Functions, is on-going in ITU-R WP6C (RG-24) and in SMPTE 10-E as described later in this report. MPEG is currently investigating the effectiveness of existing compression technologies with high dynamic range video, and the means to support standard and high dynamic range video within a single coded representation BT.709 with 8 bit code levels BT.709 with 10 bit code levels DICOM Part 14 Perceptual Quantiser, SMPTE ST 2084:2014 BBC HDR Proposal - 10 bit code levels Code Levels Figure 8. The mapping of linear light to code levels. 3.5 The mapping of pixel code levels back to linear light While Cathode Ray Tubes (CRT) were the only or most popular display device for television, there was no need to specify an inverse to the OETF, known as the Electro-Optical Transfer Function (EOTF), as the inverse function was provided implicitly from the physics of cathode ray tubes. Fortunately, this function was well matched to the human visual system. It was only in 2011, by which time the consumer market for CRTs had almost completely disappeared, that a standard for an EOTF, ITU-R Recommendation BT , was finally agreed. This effectively documents the characteristics of CRT displays. The EOTF is not necessarily best defined as the inverse of the OETF: the combination of the OETF and the EOTF yields the total transfer function, sometimes known as the end-to-end gamma or system gamma. Some believe that this total transfer function should be a power law function with the exponent value dependent on the lighting condition, for example, with values of 1.0, 1.25, and 1.5 being appropriate for bright, dim, and dark surrounding environments 19. The fundamental basis for interpreting any visual signal is the transfer function, the description of how to convert the signal, that is, the digital code values, to optical energy. It is therefore this EOTF and not the OETF that truly defines the intent of visual signal code values. The vast majority of content is colour graded (either live in the camera, or during post production) according to artistic preference while viewing on a reference standard display. 19 Report ITU-R BT , (03/2014), The present state of ultra-high definition television. Page 15

17 The EOTF, when applied to code values with a given bit depth, 8, 10 or 12 bits etc., should ideally avoid the viewing of discontinuities in tone reproduction, and hence the EOTF and bit depth should be matched to the contrast sensitivity function of the human visual system. A well respected model for the contrast sensitivity function of the human visual system was developed by Peter Barten 16, and has been referenced by many electronic imaging studies and standards. This complex model, based on physics, optics, and some experimentally determined parameters, has been shown to align well with many visual experiments spanning several decades of research. Dolby Laboratories have used the Barten model directly to compute an optimized perceptual EOTF 17. This function is defined in Figure 9. This is defined in terms of absolute luminance levels viewed on the display screen, not in terms of absolute luminance at capture. This Perceptual Quantiser curve has nearly a square root behaviour (slope = -1/2) at the darkest light levels, consistent with the Rose-DeVries law, and then rolls off to a constant zero slope for the highest light levels, consistent with the log behaviour of Weber s law. Between those extreme luminance regions, it exhibits varying slopes, and throughout the mid luminance levels it exhibits a slope similar to the gamma non-linearities of BT Y = L ( V1 1 n m c 1 c 2 c 3 V 1 ) m 0 V 1 L = m = n = c 1 = c 2 = c 3 = Figure 9. The EOTF derived from the perceptual quantiser function, as proposed Dolby. Dolby Laboratories reported 17 the results of subjective tests with real images with peak luminance up to 600cd/m 2, comparing BT.1886 scaled for a peak luminance of 1000cd/m 2 with the perceptual quantiser based EOTF scaled to 1000cd/m 2, and scaled to 10,000cd/m 2. They reported finding that with both variants of the perceptual quantiser, 10 bits of bit depth were sufficient to avoid visible quantisation steps on all eight test images, whereas the use of BT.1886 required less bits for light (white) images, typically one bit less of bit depth, but needed more bits for the dark (black) images, including needing more than 12 bits of bit depth for one image. These results are consistent with the known limitations of BT.1886 that it has greater precision for lighter regions than darker regions, and effectively wastes code values at the light end, and does not have enough at the dark end. It is generally thought that it would be difficult to operate legacy infrastructures with bit depth greater than 12 bits, and most live production and broadcast environments still operate at the 10 bit level. Hence it is commonly agreed that applying BT.1886 to a higher dynamic range is not feasible as the bit depth required would be too high. However, there is no consensus that the perceptual quantiser proposed by Dolby Laboratories is the best solution: there are concerns about its being defined in terms of absolute light levels, and other functions have been proposed, as are discussed elsewhere in this report. 3.6 Black level: how dark should displays be? How dark or black a region of a display can appear depends on two factors: the minimum emission from the display and the amount of ambient light that is reflected. The effective display black level, Lblack, can be calculated, as in the equation below, as the sum of the display minimum light emission, Lmin, known as dark current in the days of CRT, and meaning the lowest level of luminance that comes out of the display; and a product of the display screen reflectivity, Rdisplay, and the ambient light level, Eambient. The impact therefore of higher levels of ambient light is to raise the minimum black level, and consequently to reduce the dynamic range of the image, as the maximum intensity is mostly unchanged with ambient light. L black = L min + (R display E ambient ) π Mantiuk et al. 20 have reported an experiment to determine the highest luminance level that cannot be discriminated from absolute black as the surrounding luminance is varied. They showed viewers a patch in the centre of a screen with absolute black on one side of the patch and non-zero luminance on the other, and asked viewers to choose the side that was brighter, or choose randomly 20 Mantiuk, R. and Daly, S. and Kerofsky, L. (2010), The luminance of pure black: exploring the effect of surround in the context of electronic displays. In: Proc. of Human Vision and Electronic Imaging XXI, IS&T/SPIE's Symposium on Electronic Imaging. Page 16

18 Black Level (Logarithm to the base 10 of values measured in cd/m2) if they looked the same. Different values of non-zero luminance and of surrounding luminance were tested. Two viewing distances were used, 1.4m and 4.7m, so that the size of the square patch corresponded to 6.1 and 1.8 visual degrees. They converted these results of just detectable differences from absolute black so they could be plotted as a function of ambient light rather than the luminance of surrounding pixels. The results are shown in Figure 10, with the experimental results labelled as HVS Small Patch and HVS Larger Patch. It can be seen that as the ambient illumination is increased, the lowest level of black that can be distinguished from absolute black increases. Also shown in Figure 10 is the black level of a black diffuse surface with reflectance of 3%, such as black velvet, and the performance of three displays: a CRT with minimum light emission of 1cd/m 2 and 3% reflectance; a conventional CCFL (cold cathode fluorescent lamp) backlight LCD with minimum light emission of 0.8cd/m 2 and 1% reflectance; and a modern LED-backlight LCD with spatially uniform back-light dimming with minimum light emission of cd/m 2 and 1% reflectance.1.3/2.4 The CRT appears grey compared to the diffuse black (velvet) for ambient light below 300lux (about 2.5 on the horizontal axis of Figure 10), a level of brightness found in an office or a very well lit room in a home 21. For the CCFL-LCD, this threshold is 100lux (2.0 in Figure 10), typical of a room in a home. This is because the display effective black level is higher than the luminance of a diffuse black surface, due to a combination of reflectance and the minimum light emissions of the displays. The experimental results indicate that the eye can appreciate even deeper black than a diffuse black surface, and that of the considered display technologies, only the LED-LCD display can satisfy the demands of the human visual system, and only at levels below about 1.6lux (0.2 in Figure 10, where the LED-LCD curve crosses the HVS Larger Patch curve), an indoor illumination level that could be considered near pitch black. The problem is not the minimum light emissions which are very low, but the reflectance of ambient light from the screen. It appears unlikely that very low reflectance coatings will be possible, but this is unlikely to matter, as a reflectance of about 1% is likely to acceptable to almost all viewers as there are not many objects in the real-world that would have lower reflectivity and thus appear darker than a display CRT CCFL-LCD LED-LCD Diffuse Black HVS Small Patch HVS Larger Patch Deep Twilight Dark Room Home Office Ambient Luminance (Logarithm to the base 10 of values measured in Lux) Figure 10. Comparison of the black levels of displays and human detection capability. 3.7 The mapping of pixel code levels to linear light in the presence of ambient light Although standards including BT.1886 and IEC (srgb 22 ) specify the mapping of pixel code values to light levels, the actual perceived light level on a screen also depends on the ambient light and the reflectivity of the screen Page 17

19 Pixel Luminance (Logarithm to the base 10 of values measured in cd/m2) Mantiuk et al. 23 have developed a display model that combines the standardised gamma mapping with a factor to include the reflections of ambient light. This display model, shown in the equation below, includes the maximum brightness and dynamic range of the display device, and the viewing conditions, the amount of the ambient light that is reflected from the screen. Such reflected light increases the luminance of the darkest pixels shown on the display, thus reducing the available dynamic range. L d (L ) = (L ) γ (L max L black ) + L black + (R display E ambient ) π They claim that most CRT and LCD displays can be modelled with this equation, where the displayed luminance, Ld, is calculated as a function of the luma (pixel value), L, and the display gamma, γ, the peak display luminance, Lmax, the display black level, which is the luminance of a black pixel displayed in a perfectly dark room, Lblack, the display screen reflectivity, Rdisplay, and the ambient light level, Eambient. Figure 11 shows the mapping of pixel code levels to linear light in the presence of different levels of ambient light using this equation from Mantiuk et al. for a display with peak display luminance 80cd/m 2, display black level of 1cd/m 2, reflectivity of 1% and gamma of 2.2. It can be seen that the effective dynamic range of a display gets compressed due to screen reflections, making lower pixel values almost indistinguishable. The dynamic range of nearly 2600:1 of srgb is reduced to only 75:1 in a dimly lit room, and to only 6:1 in sun light srgb : Dynamic Range 2584:1 Dim Room (20 lux) : Dynamic Range 75:1 Light Room (150 lux) : Dynamic Range 54:1 Sun light (5000 lux) : Dynamic Range 6: Luma pixel value (0.005 to 1.000) Figure 11. The mapping of pixel code levels to linear light in the presence of ambient light. Mantiuk et al. addressed this issue with adaptive tone mapping. As ambient light increases, the image gets brighter to avoid dark tones, which are the most affected by the display reflections. For the outdoors illumination, many of the bright pixels are clipped to the maximum value. An example of the actual result of their tone mapping, and how it differs for different ambient light conditions, is shown in Figure R. Mantiuk, S. Daly and L. Kerofsky, Display Adaptive Tone Mapping, ACM Transactions on Graphics, Vol. 27, No. 3, Article 68, Publication date: August Page 18

20 Figure 12. An image tone-mapped for three different ambient illumination conditions. Kunkel and Daly 24 stated that current displays are very thin and highly reflective perpendicular to the display: they are similar to mirrors! The earlier LCD displays had a matte, diffuse effect, with relatively low reflectivity, but the matte caused some image blur. Glossy LCDs eliminate the blurring effect, but at the expense of more reflections, making it important to position the screen and the viewer to minimise light sources affecting the experience. While this is controllable to some extent in the home environment, it is not so for outdoor use and signage. They state that as screens get brighter and brighter, the reflection of the viewer, illuminated by the light from the display, becomes more of an issue, but screen manufacturers may be reluctant to address this issue in the short term because such a TV looks good when turned off: the current design popular for marketing. Sharp have developed technology to minimize screen reflection, which they refer to as moth eye 25. This technology, showcased in October 2012 at CEATEC, Japan's largest consumer electronics show, involves applying an anti-reflecting coating to LCD panels based on technology similar to the nanostructure of a moth s eyes: the surface of a moth's eyes is covered with bumps and valleys that absorb oncoming light, enhancing night vision. Unlike conventional anti-reflection technology, Sharp s claimed its new LCD offers more vivid colour images and higher contrast. It demonstrated 60, 70 and 80 inch moth eye panels at CEATEC based on its Aquos large-screen TVs. Sharp said its panel technology is ready for deployment in commercial products for indoor use. But Kunkel and Daly comment that although moth-eye displays are diffuse, with even lower reflectivity than the earlier matte screens, they currently suffer from not being scratch resistant. Thicker displays were demonstrated at CES Kunkel and Daly claim these allow better audio because larger speakers can be included, and that they also allow for better backlight modulation. They also comment on curved screens, stating that curvature may help with screen surface side effects, and may enhance immersion, and that curved displays may be good for mobile use as they may allow the user to more easily avoid reflections from external light sources. 3.8 The current state of high dynamic range capture and display technology Modern digital motion imaging sensors can originate linear video signals having dynamic ranges up to about 14 stops, that is, ranges up to about 16,000:1. This dynamic range is similar to the simultaneous dynamic range of the human visual system 18. Such cameras include the Arri Alexa and Amira, various models by Canon including the EOS C300 and EOS C500, the Red Epic, and various models by Sony including the A7S and the F65. Cinema5D have reported results of tests carried out on a selection of cameras 26. They used the DSC labs XYLA-21, a high quality LEDbacklit transmissive chart that displays 21 stops of dynamic range: each vertical bar in the chart represents one stop of light. The chart is filmed with each camera in turn in a completely dark room using the same very sharp Zeiss 50mm CP2 T/2.1 makro lens with interchangeable mount adjusted for the camera bayonet. Each camera was set to its native ISO and the F-stop of the lens was 24 Timo Kunkel and Scott Daly, Dolby Labs, Inc., SMPTE Monthly Webcast: Lessons in Light: From Reality via Display to the Eye Page 19

21 adjusted accordingly. The Intra frames were extracted from the recorded video and tested with software from IMATEST. The results are reproduced in Figure 13. It can be seen that the Arri and Sony cameras produced a usable dynamic range of about 14 stops, and the Canon cameras about stops. Figure 13 also shows that these cameras use a variety of transfer functions to map linear light to digital code values. The RED Epic with a Mysterium-X sensor is claimed to support 13.5 stops of dynamic range, and up to 18 stops when using RED HDRx 27. The Panasonic VariCam 35 is claimed to support 14+ stops of dynamic range 28. Dynamic range is considered more important to some camera manufacturers than increased spatial resolution. For example, Arri has stated 29 that it will not move to 4K until it can do so without compromising the dynamic range on its range of cameras, including the Alexa and the newly introduced Amira. They believe that they have the best dynamic range on the market today, and that there is interest from the creative community to create better pixels, not just more pixels. The Alexa is among the most widely-used digital cinematography cameras. Figure 13. Camera dynamic range measurements by Cinema5D. Miller et al 17 of Dolby Laboratories report that typical displays today are now achieving peak levels of 500cd/m 2 or more, and that there are several commercial examples above 1000cd/m 2. As an example, the Samsung OL46B - OL Series 46" Outdoor High Brightness High Definition Display 30 is stated to have a peak brightness of 1500cd/m 2. However, typical televisions appear to have peak brightness levels up to about 350cd/m 2. For example the Toshiba 48" LED-backlit LCD HDTV, 48L1435DB 31, has a brightness of 300cd/m 2 ; and the Sharp 50" LED-backlit LCD HDTV, LC 50LE651K 32, has a brightness of 350cd/m 2. MPEG plans to use two high dynamic range displays in analysis of the submissions to its Call for Evidence of new tools that may improve the performance of HEVC when used to encode high dynamic range and wide colour gamut video. Subjective testing will be done using a SIM2 monitor in two locations and a Pulsar monitor in another location GGH.html?refs= &q=brightness&src=16#usedstock 32 Page 20

22 The sim2 33 is a 47inch HD display with 2202 individually controllable white LED backlights, peak brightness of 4000cd/m2 and BT.709 colour gamut. It supports two input modes: HDR Yu v where 12 bits are assigned for Y in a log representation, and 10 bits are assigned for u and v using linear encoding, with 4:2:2 sampling; and DVI plus mode, in which the input comprises an LED signal (2202 values) and the LCD signal (1920x1078, 8 bit RGB values). This display previously had an issue when using the HDR Yu'v' input mode, relating to chroma sub sampling and different sample locations. This issue has now been fully resolved, and the display has been approved for use by the MPEG community. The Dolby Pulsar has been used for demonstration and experimental purposes 12. It is a 42inch HD display with peak brightness of 4000cd/m 2, DCI-P3 colour gamut, and 12 bits per colour sample 34. It is likely that higher dynamic range televisions will not reach the market as a step change, but that over time, televisions with increasing peak brightness will become available. As stated above, televisions are already available that support peak brightness levels up to about 350cd/m 2. It is possible that this could steadily increase to 1000 cd/m 2 and beyond. But a change in broadcast or storage media format will be necessary for these screens to benefit from true higher dynamic range rather than just increased brightness. 3.9 Interest and Experience in Hollywood There s a growing number in Hollywood who want high dynamic range, which, as it is independent of the number of pixels in a frame, can be used with any picture resolution. There is a growing number of industry veterans that say, given a choice of high dynamic range or ultra high definition, it is high dynamic range that creates the more noticeable improvement, and hence it is this option that is generating a lot of interest 29. Emmanuel Chivo Lubezki, director of photography of the movie Gravity, is reported to have stated, I think every cinematographer will have an interest in high dynamic range, and that he would explore this potential for all his future projects 31. A short film, entitled Emma was shot in high dynamic range by director Howard Lukk and cinematographer Daryn Okada. The first four minutes of the 13 minute film was shown at the SMPTE 2014 Symposium, October An Alexa camera was used in Open Gate mode 36, capturing at a resolution of 3414x2198, and capturing into the ARRIRAW format using the ACES colour space. It was reported that one of the biggest problems on set was monitoring: they were shooting a high dynamic range movie and using a standard dynamic range monitor. Consequently they had to rely on a light meter to estimate light drop-off in certain shots. Shooting in high dynamic range is reported to have added no extra time, but spending more time with the make-up artist would be beneficial. With high dynamic range it is possible to see so much more, not only in the actors faces but also details in the background, which might not be desirable. With high dynamic range it was reported to be more difficult to hide things and hence they spent more time on masks to get the right compositions. On the subject of whether it was possible to convert previously shot projects into high dynamic range, it was reported: If you shot on a very good dynamic range element, meaning film, you can probably pull out a high dynamic range out of it. If you re trying to go back and pull it out of a video camera from 10 years ago it will be impossible Better Pixels: Color Volume and Quantization Errors, Robin Atkins, Dolby Labs Alexa XT Open Gate. Record it all: Open Gate sensor mode. Page 21

23 It was reported that the following would help with future high dynamic range productions: Finding a high dynamic range display to use on screen for monitoring Locking down a transfer function for the high dynamic range master such as ACES or OCES Settling aspect ratio, since scope on Ultra-HD has to be downsized and letterboxed (otherwise pan and scan) Settling on a standard light level 3.10 Dolby Cinema Dolby Cinema was announced 37 in December It will feature the Dolby Vision projection system, which is claimed to be able to deliver high dynamic range with enhanced colour: its amazing contrast, high brightness, and colour range closely matches what the human eye can see. They claim that the blacks are truly black, colours are vibrant, and the contrast ratio far exceeds that of any other image technology on the market today. The first Dolby Vision projectors and titles for theatrical exhibition are expected in Early Dolby Cinema locations will be equipped with high-brightness laser projection technology available today with 4K, high-framerate 2D and 3D capabilities. When Dolby Vision content becomes available, the Dolby Cinema laser projection systems will be replaced with Dolby Vision projection systems Page 22

24 4 Wider Colour Gamut The human visual system is sensitive to electromagnetic radiation in the approximate range of wavelengths from 400nm (blueviolet) to 700nm (red), using three types of cone cell that have different spectral response. 4.1 The CIE RGB Colour Space In the 1920s, W. David Wright and John Guild independently conducted a series of experiments on human colour vision. The experiments used a circular screen, of size two degrees to the observers, showing on one side a single wavelength test colour and on the other side a viewer adjustable mix of three primary colours, red, green and blue. The viewers were asked to adjust the mixture of primary colours until they considered a match with the single wavelength test colour to have been achieved. This was not possible for all test colours, where instead a variable amount of one of the primaries was added to the test colour, and a match with the remaining two primaries was carried out. The CIE RGB colour matching functions were consequently defined using these experimental results, using three monochromatic primaries at standardized wavelengths of 700nm (red), 546.1nm (green) and 435.8nm (blue). The primaries with wavelengths 546.1nm and 435.8nm were chosen because they are easily reproducible monochromatic lines of a mercury vapour discharge. The 700 nm wavelength, which in 1931 was difficult to reproduce as a monochromatic beam, was chosen because the eye's perception of colour is rather unchanging at this wavelength, and therefore small errors in wavelength of this primary would have little effect on the results. The colour matching functions are the amounts of primaries needed to match the monochromatic test colour. These colour matching functions, known as the "1931 CIE standard observer", are shown in Figure 14. The wavelengths at which one of the colour matching functions goes negative correspond to the test cases where a variable amount of one of the primaries was added to the test colour, and a match with the remaining two primaries was carried out: the test colour is effectively being matched by a positive contribution from two primaries and a negative contribution from the other R(λ) G(λ) B(λ) Wavelength (nm) Figure 14. The CIE 1931 RGB Colour Matching Functions. The RGB tristimulus values, (R, G, B) for a colour with spectral power distribution, I(λ), can be calculated as the integral of the product of the colour matching function (R(λ ), G(λ ), B(λ)) and the spectral power distribution, I(λ ), as shown below. Note that many different spectral power distributions correspond to the same RGB tristimulus values, and also appear to the human visual system to be the same colour. R = 0 I(λ)R(λ)dλ G = 0 I(λ)G(λ)dλ B = 0 I(λ)B(λ)dλ Page 23

25 The chromaticity of a spectral power distribution, (r,g), can be calculated by normalising the RGB tristimulus values: r = R R + G + B, g = G R + G + B This results in the CIE rg chromaticity space as shown in Figure 15. The curve shown represents the colours of single wavelength. Colours within this locus can only be made by mixing light of different wavelengths together, including those colours that lie along the line g=0. Points outside of the locus are not real colours and cannot be observed. 2.0 Green 1.5 g Blue r Red 4.2 The CIE XYZ Colour Space Figure 15. The CIE rg Chromaticity Diagram. The CIE decided 38 to map the RGB colour space onto a new derived colour space, XYZ, with derived colour matching functions, and derived chromaticity space, with the benefits that the derived colour space would satisfy the following constraints: The colour matching functions would be positive or zero at all wavelengths; and The Y colour matching function would be exactly equal to the photopic luminous efficiency function V(λ) defined by the CIE in 1926 for the "CIE standard photopic observer ; The RGB colour space is mapped onto the XYZ colour space using the following linear transformation, which has been standardised by the CIE 39 : X R [ Y] = ( ) [ G] Z B 38 How the CIE 1931 Color-Matching Functions Were Derived from Wright Guild Data, Hugh S. Fairman, Michael H. Brill, Henry Hemmendinger, Matching%20Functions%20Were%20Derived%20from%20Wright-Guild%20Data.pdf 39 Publication CIE No. 15.2, Colorimetry, Second Edition, Central Bureau of the Commission Internationale de l Éclairage, Vienna, Austria, Page 24

26 As described above for the RGB colour space, and shown in the equation below, the chromaticity of a spectral power distribution, (x,y), can be calculated by normalising the XYZ tristimulus values. And when this is done, by the requirement of positive values of X, Y and Z, the gamut of all colours lies inside the triangle [1,0], [0,0], [0,1]. x = X X + Y + Z, y = Y X + Y + Z The CIE 1931 xy chromaticity diagram, taken from Wikipedia 40, is shown in Figure 16, together with the RGB primaries: 700nm (red), 546.1nm (green) and 435.8nm (blue). All colours within this triangle defined by the three primaries can be made by mixing the three primaries in the appropriate non-zero combination. Colours outside of this triangle can be made by mixing the primaries, but only if negative quantities are allowed. This is why in the Guild and Wright experiments, some single wavelength test colours had to be colour matched by mixing the test colour with one primary, and matching against a mixture of the other two primaries. Both the XYZ and the xyy colour spaces can be used in practice. The latter consists of the pure chromaticity values x and y, and the luminous value Y. 4.3 Perceptually Uniform Colour Spaces Figure 16. The CIE 1931 xy chromaticity diagram showing the CIE RGB primaries. The CIE 1931 XYZ colour space is not perceptually uniform, that is, a constant magnitude change in the tristimulus values does not correspond to a constant perceptual change. The CIE has developed colour spaces that are closer to being perceptually uniform. These include the Lab and Luv colour spaces standardised by the CIE in Both of these colour spaces are derived from the master XYZ colour space, with dimension L for lightness and a and b or u and v for the colour dimensions. Both share the same definition of lightness, L, with L=0 indicating black, L=100 indicating diffuse white, and higher values indicating specular highlights, but they have different definitions for the colour dimensions. It is believed that Lab is the more widely used, and that both were standardised as the standardisation committee could not reach agreement on a single one Page 25

27 4.4 Colour Television Colour television has taken advantage of the above observations on the mixing of primary colours: by choosing three primary colours, and mixing them appropriately, all colours within the triangle on a chromaticity diagram defined by the three primary colours can be created. Cathode ray tube displays, the primary means of watching colour television until relatively recently, use three different phosphors which emit red, green, and blue light respectively, when excited by one of three beams of electrons. LCD displays achieve colour by applying three colour filters, red, green, and blue, to the backlight. In both cases of CRT and LCD displays, the colour primaries are not single wavelength sources, but have quite spread spectra, as shown in Figure 17, where the CRT image is from Wikipedia 41 and the right image is recreated from Chen at al Wavelength (nm) Figure 17. The spectra of display primary colours, typical CRT on the left, typical LCD with colour filters on the right. The spectra of the colour primaries of display devices limit the range (gamut) of colours that can be shown on the display. The ITU-R Recommendation BT.709, and the earlier BT.601, defined a set of colour primaries, in terms of the CIE 1931 x and y chromaticity diagram, to be used in television systems. These colour primaries, and the colour space defined by them, are shown in the CIE 1931 xy chromaticity diagram of Figure 18, taken from Wikipedia 43. This system is still in use today, although as described fully below, there is significant interest in extending display specifications and consequently producing devices to support wider colour spaces. It can be seen clearly in Figure 18 that this colour space leaves a large set of colours that could not be displayed. Care should be taken when quantifying this, because, as pointed out above, CIE 1931 is not a perceptually uniform colour space Quantum-Dot Displays: Giving LCDs a Competitive Edge through Color, Jian Chen, Veeral Hardev, and Jeff Yurek Page 26

28 Figure 18. The primary colours of the BT.709 colour space, and its primary colours on the CIE 1931 xy chromaticity diagram. 4.5 The need for a Wider Colour Gamut Current television systems do not support the wide range of colours that the human eye can perceive. Future television and other video distribution environments are hoped to give a viewing experience that is closer to a real life experience, to provide a user with a stronger sense of being there. This requires supporting both significantly higher dynamic ranges and wider colour gamuts than supported today. As stated above, Figure 18 shows the primary colours of the BT.709 colour space. These are joined to form a triangle, which is often referred to as the bounds of the colour space. But this is only true for low values of luminance. For higher values of luminance, the range of colours is reduced, such that for the maximum relative luminance of 1.0, the chromaticity gamut is only a single point, being the white point of the colour space on the chromaticity plane 44. Hence in order to describe the full range of colours that can be represented in a colour space, for example that described in BT.709, it is necessary to consider a 3D colour volume. Assuming this 3D colour volume to be drawn with the x and y chromaticity coordinates in the horizontal plane, and the luminance, Y, vertically, then the horizontal cross section of the 3D colour volume is the triangle shown on Figure 18 up to a relative luminance of , at which point the blue primary cannot add to the relative luminance. In order to increase relative luminance of blue beyond this level, it is necessary to desaturate it by adding red or green or a mixture of the two. Hence from upwards, the cross section becomes a quadrilateral, with two vertices directly above the red and green primaries, and with two vertices inward from the blue primary, to an increasing extent as relative luminance is increased. At relative luminance of , the red primary similarly cannot add to the relative luminance, and hence to increase relative luminance of red beyond this level, it is necessary to add blue or green or a mixture of the two. The cross section consequently becomes pentagonal, but only until relative luminance reaches = , at which the blue plus red vertex meets the red plus blue vertex, and the luminance of either can only be increased by adding green, and the cross section returns to being a quadrilateral. At a relative luminance of , the green primary can no longer add to the relative luminance on its own, and the cross section again becomes pentagonal. As relative luminance increases further, the cross section becomes quadrilateral yet again, and finally triangular at relative luminance of , reaching an apex at relative luminance of , and the D65 white point given by chromaticity coordinates (x=0.3127, y=3290). This is shown on the CIE 1931 xy chromaticity diagram in Figure 19, where in addition to the showing the spectral locus, labelled as CIE 1931, the cross section of the BT.709 3D colour volume is shown for various values of luminance, showing that the cross sectional area reduces as the relative luminance increases Page 27

29 Y y CIE 1931 Y = Y = Y = Y = Y = Y = Y = x Figure 19. The horizontal cross section of the BT.709 3D colour volume for various values of luminance. Figure 20 shows another view of the 3D colour volume, this time showing a vertical cross section, with chromaticity coordinate, x, plotted horizontally and the relative luminance, Y, plotted vertically, for chromaticity y = The colours shown on the figure are illustrative, and clearly not precise. But it is can be seen, particularly on the right hand side, that as the relative luminance is increased, the range of colours that can be represented is reduced Out of Gamut Colour x Figure 20. A vertical cross section of the BT.709 3D colour volume for y = Page 28

30 This illustrates the need for both a wider colour gamut and a higher dynamic range, as stated by Kunkel and Daly 24, who quote an example of a volcano at night, such as that illustrated in Figure Emissive colour, such as the molten lava of Figure 21, can be both very bright and very saturated, and outside of the 3D colour volume, as shown on Figure 20. Without a change to colour gamut or dynamic range, the orange of the volcano would have to be moved into the colour volume. This could be accommodated by making it darker, effectively moving it down in Figure 20 so that it moves into the colour volume or by making it less saturated, effectively moving it horizontally into the colour volume, or some combination of both. When using a higher dynamic range and a wider colour gamut, the colour volume increases in all three dimensions, allowing such a point to be represented: just having one of these aspects may not be enough. The shape of the colour volume is such that at high luminance, it is not possible to have high saturation: while a display may be able to display white with 1000cd/m 2, it may not be able to display green with 1000cd/m 2 because of the way the display is built. Combining a wider colour gamut with a higher dynamic range will create a larger colour volume which can then transport more colours. 4.6 Wider Colour Gamut Standards Figure 21. Emissive colour can be both bright and saturated. The colour space defined in BT.709 has been widely adopted, and is common to other standards, including BT.1361, IEC (srgb or sycc), IEC , and SMPTE RP 177 (1993) Annex B. The DCI P3 colour space is used by digital cinema. It is a part of the SMPTE Recommended Practice RP for Digital Cinema Quality Reference Projector and Environment, first released by the SMPTE in 2007, and most recently updated in It defines the Reference Projector and its controlled environment, along with the acceptable tolerances around critical image parameters for Review Room and Theatre applications. Although the colour gamut is specified in terms of XYZ, a Digital Cinema projector is required to support a gamut of at least DCI-P3. The DCI-P3 colour space uses the same blue primary as BT.709, but it uses different green and red primaries. The red primary of DCI- P3 is monochromatic 615 nm, a slightly deeper hue of red than BT.709. The green primary is a slightly more yellowish hue of green but is more saturated. In CIE 1931 (x,y) coordinates, the DCI-P3 primaries are red (0.680, 0.320), green (0.265, 0.690) and blue (0.15, 0.060). Additional colour spaces with wider gamut have been defined with the aim of supporting wider colour gamuts in the future. The ITU-R has specified a colour space in BT.2020 using single wavelength primaries. In addition, specifications have been defined that cover the full visible gamut. For example, SMPTE specified in ST usage of the CIE 1931 XYZ colour space, and the Academy Color Encoding System (ACES) defined the ACES colour space, which is similar to XYZ, but slightly smaller while still encompassing the whole spectral locus. In CIE 1931 (x,y) coordinates, the ACES primaries are red ( , ), green ( , ) and blue ( , ). Figure 22 shows the BT.709, DCI-P3, BT.2020, ACES and XYZ colour gamuts on the CIE 1931 xy chromaticity diagram Page 29

31 4.7 Conversion between Colour Gamuts As colour gamuts are specified relative to CIE standards, conversion between colour gamuts is possible. When converting from one colour gamut to another, where the target gamut fits fully within the originating gamut, is straightforward, being essentially a mathematical transformation. For example, as described later in this report, the ITU-R is understood to be developing a new Recommendation, to specify a method of colorimetry conversion from BT.709 to BT Such a conversion could be needed for example when HDTV programme content is included within UHDTV programmes. However, other conversions, for example from BT.2020 to BT.709, are more problematic as there may be colours in the originating colour space that cannot be represented in the target colour space. These out of gamut colours cannot be represented accurately, and so some loss in quality must be tolerated. The problem is that while out of gamut colours need to be changed to bring them into the gamut, colours that are not outside the gamut do not need to be changed, and the larger any change to them is, the more degradation occurs, but if they are not changed, they will no longer have an appropriate relationship to colours that are moved into the gamut. A high quality gamut mapping algorithm should minimise the change of colours in the original representation that are within the gamut of the target colour space, minimise the perceived changes of hue, minimise changes to brightness, contrast and saturation, avoid loss of spatial detail, and avoid creating visible discontinuities. Frohlich et al 46 report that colour gamut conversion is a problem today in the cinema industry where newer digital cinema projectors are capable of a wider colour space than older projectors, and where content is now being produced for these newer projectors, but where the issue of supporting older projectors with a narrower colour space remains. Current practice consists of simply clipping each colour component (RGB) to fit within the target colour space, but this is reported to result in hue shifts and loss of detail. Frohlich et al take the common view that a perceptually uniform colour space is best suited for performing such a conversion: a perceptually uniform colour space is one in which equal (Euclidean) distances in the colour space correspond to equal perceived colour differences. But rather than using the popular CIE L*a*b* uniform colour space, they prefer one defined by Ebner and Fairchild 47 known as IPT. They report the results of subjective tests comparing algorithms that they refer to as PCLIP and WmindE. PCLIP, short for Post-CLIP is assumed to be implemented in projectors today, and consists of simple clipping of each colour component (RGB) in the projector s native colour space. This is reported to result in hue shifts and loss of detail. The tested alternative, WmindE, which is short for weighted minimum delta E, selects the in-gamut colour with the minimum colorimetric distance for each out-of-gamut colour, but where a different weight is used for each of the components, lightness, chroma and hue. They report that in their subjective tests, WmindE in the IPT colour space was found to offer better detail preservation than PCLIP while retaining hue; and that WmindE was preferred over PCLIP by 80% of the viewers. They claim that all of the algorithms that they tested except PCLIP are too complex to render in real-time in current projectors, but that they could be implemented in a 3D lookup table. 4.8 How large a Colour Gamut is needed? While the chromaticity diagrams referred to above indicate the full gamut of colours that the human visual system can perceive, it is difficult if not impossible to produce displays that show the full gamut. Particularly, if displays are limited to three primary colours, the largest gamut that could be created is one defined by the largest triangle that could be placed within the chromaticity diagram. But there is still a question of how large the gamut needs to be, to provide a realistic viewing experience. The work of Pointer 48 addresses this question. A maximum gamut for real surface colours, known as Pointer s Gamut, was derived from the analysis of the colour coordinates of 4089 samples of real surfaces. Figure 22 shows Pointer s Gamut of real colours on the 1931 CIE xy chromaticity diagram, and the standardised colour spaces, BT.709, DCI P3, BT.2020, ACES and XYZ. It can be seen that while BT.709 and even DCI P3 do not cover the whole of Pointer s Gamut, 46 Jan Frohlich, Andreas Schilling, and Bernd Eberhardt, "Gamut Mapping for Digital Cinema," SMPTE Conf. Proc. vol. 2013, no. 10, pp (2013). 47 F. Ebner and M. Fairchild, Development and Testing of a Colour Space(IPT) with Improved Hue Uniformity, in the proceedings of the Sixth Color Imaing Conference: color Science, Systems, and Applications, page 8-13, The Gamut of Real Surface Colours, M. R. Pointer, Color Research & Application, Volume 5, Issue 3, pages , Autumn (Fall) 1980 Page 30

32 y BT.2020 does in fact cover 99.9% of it 49. Hence, although there are many colours that could not be described in BT.2020, there are very few naturally occurring colours that could not be described. So BT.2020 would appear to be a sufficient colour space for television systems to display realistic images CIE 1931 Pointer's Gamut BT.709 and srgb DCI P3 BT.2020 ACES XYZ x Figure 22. Comparison of Pointer s Gamut of real colour and standardised colour spaces. 4.9 Display Technology for Wider Colour Gamut LCD displays produce images by selectively filtering a backlight. Many individual LCD shutters, arranged in a grid, open and close to allow a metered amount of the light through, each shutter being paired with a coloured filter to remove all but the red, green or blue portion of the light from the original white source. The shade of colour is controlled by changing the relative intensity of the light passing through the LCD shutters 50. The colour gamut that can be produced by a conventional CCFL (cold cathode fluorescent lamp) backlight LCD is dependent on the spectrum produced by the backlight and the bandwidth of the filters used to produce the primaries. The gamut can me made larger by using narrower band filters on the backlight. However this also lowers screen luminance by decreasing the proportion of the backlight that passes through the filters. Increasing the luminance of the cold cathode to counter this effect tends to shorten the life of the device and often results in lighting irregularities 51. Wide gamut cold cathode fluorescent lamps are reported to be under development by many manufacturers, but not yet widely adopted because of the shorter lifetime and lower optical efficiency 52. The use of white LED backlighting is now popular. Typically, white light is produced by using blue LEDs with YAG (yttrium aluminium garnet) phosphor. Such white light typically has a spectral peak in blue, from the LED, and a broad peak around yellow from the phosphor. After red, green and blue colour filtering, with relatively wide band filters to maintain a reasonable amount of brightness in the red and green primaries, the result is usually a smaller colour gamut than CCFL backlit displays. However, such displays can Five-Primary-Color LCDs, Hui-Chuan Cheng, Ilan Ben-David, and Shin-Tson Wu, Journal of display technology, Vol. 6, No. 1, January 2010, Page 31

33 achieve high dynamic range, alternatively referred to as high contrast, by dimming the LEDs in the dark areas of the image being displayed. RGB LED backlighting can also be used. Small groups of pixels in the LCD panel are backlit using triads of controllable red, green and blue LEDs. This increases the display s colour gamut and the bit depth, as well as providing deep black levels due to localised dimming of the backlight 53. The Dolby Professional Reference Monitor PRM-4200 is an example, using dual modulation technology, where the backlight unit consists of approximately 1,500 RGB LED triads, modulated on a frame-by-frame basis, that directly illuminate the LCD panel 54. In 2013, it was reported 55 that this was the only commercially available monitor capable of displaying the DCI P3 colour gamut. However since then other products have been announced. Another means to widen the gamut is to use more than three colour primaries. This has been demonstrated in projection displays and some direct-view LCDs. However, the display brightness is reduced and cost is increased because of the increased number of colour pixels and fabrication complicities 52. Quantum dots is another backlight technology, where blue LEDs are used, but instead of using a phosphor to produce white light, a layer of quantum dots is used. These act in a similar way to a phosphor, absorbing the blue light and emitting light with a different spectrum, which is determined by the size of the nanostructures. This allows narrow bandwidth light at the desired colour primary frequencies to be produced. Jeff Yurek of Nanosys, one of a number of organisations developing quantum dot technology, has reported that it is practical to produce a display that covers over 97% of BT.2020 using quantum dot technology, and has claimed to demonstrate over 91% BT.2020 just by modifying an off-the-shelf, standard LCD TV set with a specially tuned sheet of Quantum Dot Enhancement Film (QDEF) 56. The first TV manufacturer shipping TVs of this kind was Sony in 2013, using the trade mark Triluminos, and incorporating QD Vision s quantum dot technology Color IQTM 57. Quantum dot technology is also used in the Kindle Fire HDX Samsung Electronics, LG Electronics, the Chinese TCL Corporation and Sony showed quantum dot enhanced LED-backlighting of LCD TVs at the Consumer Electronics Show Samsung introduced an extensive range of SUHD TVs at the Samsung Forum in Monaco in February These UHD displays use quantum dot technology, referred to by Samsung as nano-crystal technology, to achieve a wide colour gamut, assumed to be consistent with DCI P3 due to their announcement of collaboration with 20th Century Fox who re-mastered multiple scenes from its film, Life of Pi, specifically for the SUHD TV. Another technology that promises to be able to achieve wide colour gamut is organic light-emitting diode (OLED) technology. Unlike LCD technology, OLED does not require backlighting to function. Light is produced by a process known as electrophosphorescence when current flows through an emissive layer of organic molecules, such as polyaniline 61. The colour of light emitted depends on the exact organic makeup of the molecules. As there is no backlight, an OLED can display deep black levels and can consequently achieve a higher contrast ratio than an LCD. Active Matrix OLED technology (AMOLED) is used in a number of smartphones, including leading models produced by Samsung. 53 Color Correction Handbook: Professional Techniques for Video and Cinema, Alexis Van Hurkman Page 32

34 Sony announced the BVM-X monitor at the Hollywood Post Alliance Tech Retreat, 9-13 February Sony claims that this is its first OLED master monitor to combine UHD resolution, high dynamic range and wide colour gamut display. Sony claims that this 30-inch monitor has the ability to display ITU-R BT.709 and DCI-P3 colour gamuts more accurately than any previous Sony Trimaster display, and in addition, can display 80% of the ITU-R BT.2020 colour gamut. The Mitsubishi launched the LaserVue Series of DLP TVs in that were powered by lasers and which they claim was able to achieve 91.4% of the BT.2020 colour gamut. They also claim that of as of 2014, no other commercially available display has been able to achieve such a wide colour gamut. However, the displays, which retailed around $6000 were discontinued in December 2012 due to the inability to compete with the shrinking size and prices of LCD TVs. Displays that rely on a laser backlight system are understood to be very power-hungry and consequently impractical, and considering the European Commission eco-design directive, which is discussed further later in this research paper, unlikely to be legal in Europe due to low power efficiency. NHK laboratories are believed to have developed a laser projector, which is believed to be the only system currently capable of achieving the BT.2020 colour space, but it is likely to be very demanding of power and hence impractical for home use Page 33

35 5 Higher Frame Rate Motion reproduction is an important feature of television: to make a sequence of still images appear to be natural movement. When the frame rate is too low, artefacts are perceived by the viewer, which can be classified into flicker, motion blur, and jerkiness (stroboscopic effect). Flicker is a phenomenon in which the whole screen or a large area of the screen flickers due to low frame frequency. Motion blur is due to the accumulation mechanism of the capturing devices. It also happens when moving images are tracked on the screen of a hold-type display. Non-smooth reproduced motion at low frame frequencies is called jerkiness. The use of the shutter during acquisition can produce a multi-exposure-like image even at high frame frequencies. This phenomenon is sometimes called jerkiness as well. Motion blur and jerkiness depend on the object speed. The object speed (arc-degree/second) generally tends to become faster as the field of view of the system becomes wider. For example, given the same shooting angle, an object on an HDTV screen will move faster than the one on an SDTV screen, assuming that screen sizes and viewing distances are such that the extra detail of HDTV can be observed, that is, the HD screen fills a larger part of the viewer s field of view than the SD screen. Consequently high definition TV systems have stricter motion portrayal requirements and UHDTV systems higher still. SDTV, HDTV, and current UHD systems are specified at frame rates up to 60 frames per second. However, it is claimed that the UHDTV viewing experience could be improved by the use of higher frame rates. 5.1 The artistic impact of frame rate The choice of frame rate may sometimes be determined by whether the scene contains normal or fast movement, but it is more frequently due to the director of photography s selection of a programme look : low frame rates are often used to create a film like look, regardless of the speed of movement or action in the scene. High frame rates are often used to obtain a more contemporary look with increased motion clarity, and/or to obtain sharper slow motion images. The cinema industry still favours 24 frames per second. Daly et al. 65 comment on this, stating that in addition to the historical association of motion judder with cinema, the motion blur associated with this frame rate allows the viewer s attention to be directed to parts of the scene in a similar way to the use of a shallow depth of field; motion blur is useful for hiding cinema craft (e.g., fake beards); and aids the suspension of disbelief, as realistic imaging could hinder the viewer s imagination. 5.2 Historical choices of frame rate In the early days of television, it was considered sufficient 66 for the frame rate to be high enough merely to exceed the threshold for apparent motion, above which a sequence of recorded images appear to the eye as containing moving objects rather than being a succession of still photographs. Another factor taken into consideration was that the frame rate should be high enough that flicker was imperceptible on contemporary televisions. However, priority was not given to the elimination of motion artefacts such as smearing and jerkiness, possibly as contemporary technology may have limited the benefits of a higher frame rate anyway The motion blur jerkiness trade-off Current television frame rates cause problems for motion portrayal. Stationary objects are sharp, provided they are in focus, but objects that move with respect to the camera smear due to the integration time of the camera s sensor. Shuttering the camera to shorten the integration time reduces the smearing, but the motion breaks up into a succession of still images, causing jerkiness. The perceptual difference between moving and stationary subjects is increased with the increasingly sharper images due to new television systems with successively higher spatial resolutions, so long as the temporal resolution remains unchanged. The problems smearing, jerkiness or a combination of the two, are more noticeable with larger displays where the eye tends to follow the motion across the scene. 65 "A Psychophysical Study Exploring Judder Using Fundamental Signals and Complex Imagery", Scott Daly, Ning Xu, Jim Crenshaw, and Vickrant Zunjarrao, SMPTE Annual Conference, Zworykin, V. K. and Morton, G. A., Television: The Electronics of Image Transmission. Wiley. New York. 67 BBC White Paper WHP 209: Higher Frame rates for more Immersive Video and Television, R.A. Salmon, Mike Armstrong, Stephen Jolly. Page 34

36 Figure 23, copied from the BBC White Paper , shows the problem in terms of the movement of a ball across a plain background. In the top illustration, the trajectory of the ball is shown as if captured by a video camera with a very short shutter: each frame would show the ball frozen in time, and the motion would appear jerky when the video sequence was replayed. In the middle illustration, the effect of a half-open shutter is depicted: camera integration smears the motion of the ball out over the background, removing any spatial detail and making it partially transparent. These effects would be clearly visible in the final video sequence. The bottom image shows the effect of doubling the frame rate: both the smearing and jerkiness are reduced. A substantial further increase in frame rate would still be required in this example to eliminate their effects. Figure 23. The effects of frame rate and shuttering on motion portrayal. Without a change in frame rate and shutter speed, it takes only a small amount of motion, estimated by the BBC as three pixels per displayed image, to lose the benefits of HD over SD resolution. The problem becomes more severe as resolution increases to UHD. Just as shuttering in the camera reduces the extent of smearing, a sample-and-hold characteristic in the final display increases it in a directly comparable fashion. This smearing arises with motion that is tracked by the viewer, where the viewer s eye is following the object across the screen, but where within each displayed image the object remains stationary for duration of the frame or field. LCD televisions have this characteristic, which is one reason why these displays have a reputation for representing fast-moving material, such as sport, poorly. LCD television manufacturers have added functionality to perform a motion-compensated frame rate increase, which ameliorates the problem to some extent at the cost of introducing other artefacts when the motion becomes too hard to predict, and during cuts and cross-fades. 5.4 Subjective evaluation of moving picture quality Emoto et al 68 have reported the results of experiments using a high-speed HD camera and projector that operate up to 240 frames per second, using typical television content including sports scenes. The content was presented at 60, 120 and 240 frames per second and viewers were asked to score on the five grade quality scale. The content was presented on a 100inch display, and the 60 viewers observed from 3.7m from the screen, that is, about three screen heights away. The results, as illustrated in Figure 24, show that the scores increase as the frame frequency increases, while the extent of the improvement depends on the content. The increase in scores when going from 60 to 120 frames per second range from 0.14 to 1.04, while the average increases across all of the content are 0.46 from 60 to 120 frames per second, and 0.23 between 120 and 240 frames per second M. Emoto and M. Sugawara, Flicker perceptions for wide-field-of-view and hold-type image presentations, IDW'09, pp , 2009 Page 35

37 Figure 24. Results of subjective evaluations of picture quality at higher frame rates. The Broadcast Technology Future s Group (BTF) 69, a collaboration of the EBU, IRT, BBC, Rai, and NHK, looked at benefits of higher frame rates. This work used five test clips (train, bike, runner, football, and carousel, and shown in Figure 25), filmed at 240 fps with 100% shutter, and created lower frame rate versions from these, with different simulated shutter settings. The lowest frame rate was 30 fps. A 55 inch 1080p display was used as a higher resolution display would not support the higher frame rates. 26 viewers observed the content at 60, 120 and 240 frames per second at a distance of 3H, the point at which the 1080 HD pixels can just be discerned. The BTF reported a statistically significant improvement for all the test sequences when the frame rate was increased from 60 fps to 120 fps and when it was increased from 120 fps to 240 fps. The results averaged over the five test clips are shown in Figure 26. The difference between 60 fps and 120 fps was one grade, and the difference between 60 fps and 240 fps was nearly two grades. The BTF stated that there was some perceptible improvement in using a shorter (50%) shutter, but the results were not conclusive Page 36

38 Figure 25. BTF Test Material. Figure 26. BTF Average Experimental Results. Kuroki et al. 70 report results on the subjective impact of frame rate, where motion images from a high-speed camera and computer graphics were shown on a high-speed display. The camera captured at 1000 frames per second, and the resulting images were processed to simulate various frame rates and shutter speeds. A 480Hz CRT display was used to present the content. Viewers were asked to evaluate the difference in quality between motion images at various frame rates. Their results show that a frame rate of 120 frames per second provides good improvement compared to 60 frames per second, and that the maximum improvement beyond which evaluation is saturated is found at about 240 frames per second for representative standard-resolution natural images. Nolan described 71 a theoretical approach to frame rates and the human visual system, combining sampling theory and the human contrast sensitivity function model, for viewing content at 1.5 picture heights from the display, for tracked and non-tracked motion. When using a 100% shutter, Nolan reported that to match the temporal resolution to the spatial resolution, assuming classical sampling, for UHD content, the frame rate required was 140 frames per second for non-tracked motion and 700 frames per second for tracked motion. But it was also noted that an acceptable balance of motion blur and strobing is likely to be possible at a lower frame rate, and that there is still a need to understand the right balance between the visibility of blur (tracked) and strobing (nontracked) for a 100 fps frames per second system. 70 A psychophysical study of improvements in motion-image quality by using high frame rates, Y. Kuroki, T. Nishi, S. Kobayashi, H. Oyaizu, and S. Yoshimura, Journal of The Society for Information Display, Volume 15, Issue 1, pages 61 68, January "The Human Visual System and High Frame Rate Television", Katy C. Noland, UHDTV: Voices & Choices, 25th November 2013 Page 37