Development of Program Production System for Full-Featured 8K Super Hi-Vision

Development of Program Production System for Full-Featured 8K Super Hi-Vision Daiichi Koide, Jun Yonai, Yoshitaka Ikeda, Tetsuya Hayashida, Yoshiro Takiguchi, and Yukihiro Nishida Test satellite broadcasting of 4K and 8K Super Hi-Vision began in August 2016. We have been researching a program production system for full-featured 8K Super Hi-Vision. The system adopts a frame frequency of 120 Hz that provides higher video quality and is being developed with the aim of shooting the Tokyo 2020 Olympic and Paralympic Games. Various pieces of 8K/120 Hz equipment have been developed, including a compression recorder, a simple production switcher, a waveform monitor, a 17-inch compact display, color grading equipment, and a 120 Hz time code generator/. We have also constructed a test production system, in which the developed equipment is connected via an ultrahigh-definition signal/data interface () on a single optical cable with a video data rate of 144 Gbps. The system has been used for test production of 8K/120 Hz content. The compatibility of the newly developed system with the conventional 60 Hz system was also confirmed, thus demonstrating the prospects for a smooth transition from the current 60 Hz production system to the 120 Hz system. 1. Introduction Test satellite broadcasting of 8K Super Hi-Vision (referred to simply as 8K below) began on August 1, 2016, and work has been moving forward toward the regular broadcasting in 2018. Looking beyond that, we have been doing R&D on implementing full-featured 8K. In addition to a 120 Hz frame frequency and wide color gamut, full-featured 8K adopts a high dynamic range (HDR) for high-fidelity reproduction of tones ranging from dark to bright areas. In this paper, we explain the full-featured 8K signal and realtime transmission interface and describe various pieces of newly developed 8K equipment for program production. 2. Full-featured 8K signal and transmission interface The video format for ultrahigh-definition television has been standardized by the International Telecommunication Union-Radiocommunication Sector (ITU-R), the Society of Motion Picture and Television Engineers (SMPTE), and the Association of Radio Industries and Businesses (ARIB) 1) - 4). Of the parts that are common to the various standards (Table 1), the 7,680 4,320 pixels, the 12 bits per sample, the 120 Hz or 120/1.001 Hz frame frequency (referred to generally as 120 Hz here), the wide color gamut conforming to Recommendation ITU-R BT.2020, and the HDR are referred to as full-featured 8K (underlined in Table 1). For broadcasters or producers, this format expands the possibilities for two-dimensional video expression in TV program, while for viewers, it provides a completely new video experience 5), 6). The 8K test satellite broadcasting that has already begun uses a frame frequency of 60/1.001 Hz, but we are moving forward with research on implementing the full-featured format at the future stage of broadcasting. The data transfer rate for the full-featured 8K signals extends to 144 Gbps, and the signal is mapped to multiple optical signals of 10 Gbps each to enable transmission over Table 1: UHDTV video format (main common specifications) Item Pixel count (horizontal vertical) Color sampling structure *1 Video sampling bits Primary colors and reference white (CIE1931 *2 ) Dynamic range Specifications 120, 120/1.001, 60, 60/1.001 Hz Red (R) Green (G) Blue (B) Reference white (D65 *3 ) 8K (7,680 4,320), 4K (3,840 2,160) 4:4:4, 4:2:2, 4:2:0 12, 10 x 0.708 0.170 0.131 0.3127 SDR, HDR y 0.292 0.797 0.046 0.3290 * The case in which the horizontal and vertical pixel counts of the color difference signal are the same as for the luminance signal is indicated as 4:4:4; 4:2:2 is when the horizontal count is reduced by half; 4:2:0 is when both horizontal and vertical counts are reduced by half. * Coordinates in the color space specified by CIE (International Commission on Illumination) in 1931. * Standard white light source specified by CIE. 13

(a) (b) (c) (a) cable (b) Coaxial cable (conventional) (c) connector Figure 1: cable and connector a single cable. A multicore, multimode optical fiber interface has been developed and standardized 7) - 9). Considering recorder, a simple production switcher, a 17-inch liquid featured 8K program production, including a compression practical use, the Ultrahigh-definition Signal/Data crystal display (LCD), a waveform monitor, color grading () is designed to have about the same thickness and equipment, and a time code generator/. strength as the conventional HD-SDI metal coaxial cable 3.1 Compression recorder and about the same size cable connectors (Fig. 1). The We developed a prototype 8K/120 Hz compression interface can carry a signal a distance of about 100 m. recorder with a video input/output. The recorder is capable of recording onto a portable memory pack 3. Production Equipment for Full-featured 8K Super Hi-Vision 3 and the specifications are listed in Table 2. medium 12). The appearance of the prototype is shown in Fig. A production system used, for example, at live production The memory pack is palm-sized and has a recording sites, consists of equipment for shooting (cameras), image capacity of 6.4 TB. It can store 45 minutes of 8K/120 Hz adjustment, signal switching (routers and switchers), video by using compression data to about 1/8 and audio on a recording and playback, and signal monitoring (video monitors and waveform monitors (WFM), etc.). The resultant video signal and audio signal of a program are multiplexed for transmission to the broadcasting station (Fig. 2). For shooting, a reference camera that is capable of a frame frequency of 120 Hz and a wide color gamut was developed in 2014, and a full-featured 8K camera with the additional HDR capability was developed in 2016 10), 11). Here, we describe newly developed equipment for a full- Figure 3: 8K/120 Hz data compression recorder WFM Camera Image adjustment Monitor Camera Image adjustment WFM Monitor Signal switching WFM Monitor Transmission equipment Recording/ playback Monitor Loudspeakers Audio signal Studio Postproduction Figure 2: Conceptual diagram of the live program production system 14

Table 2: Specifications of 8K/120 Hz data compression recorder Table 3: Specifications of 8K/120 Hz blanking switcher Pixel count 7,680 4,320 Inputs/outputs 4/4 Color sampling RGB 4:4:4 Synchronization signal Black burst, Tri-level Bit depth 12, 10 bit 120/1.001 Hz, 60/1.001 Hz 120/1.001 Hz, 60/1.001 Hz Processing speed (latency) 3 scanning lines Compression ratio 1/8 Size 430 (W) 550 (D) 134 (H) mm Audio 22.2 channels (Switching and AVDL unit) Maximum transfer rate 20 Gbps Weight 17.4 kg (Switching unit) Recording medium Solid-state memory device 9.6 kg (AVDL unit) single recording medium. The memory packs are connected to the controller board of the recorder via multiple highspeed serial interfaces that allow high-speed data transfer by parallel recording of eight channels. Furthermore, a 20 Gbps recording rate is achieved by setting the data recording unit to the optimum block size for video recording to reduce the overhead of data transferred between the system board and the memory controller. The 8K/120 Hz video data is compressed in the extended JPEG format on a frame-byframe basis. A compression IP core *1 is implemented with multiple FPGA and configured for distributed processing to achieve real-time recording and playback. Special playback features such as jogging and slow-motion are also implemented. 3.2 Simple production switcher We developed a simple production switcher (blanking switcher *2 ) with (4 inputs and 4 outputs) for 8K/120 Hz signals 13). No disturbance is permitted in the output signals of the production switcher when switching video input *1 A functionally organized circuit program within an integrated circuit *2 A switching device that performs video signal switching during the vertical blanking interval Figure 4: 8K/120 Hz simple production (blanking) switcher signals from cameras or other devices. The appearance of the switcher is shown in Fig. 4 and the specifications are listed in Table 3. The structure is illustrated in Fig. 5. Optical signals of the are converted to electrical signals for simultaneous switching in the switching unit shown in Fig. 5, and then the electrical signals are converted back to optical signals for transmission. If an ordinary FPGA is used to implement the fullfeatured 8K signal routing and switching function with a data transfer rate of 144 Gbps, many high-performance FPGA devices with high-speed interfaces are required and the equipment size becomes large. To realize a practical and compact size, we employed a cross-point switching device that has 96 inputs and 96 outputs with a high-data transfer rate of 10 Gbps, which is generally used in the equipment for telecommunication. When switching video signals with a cross-point switch, however, data loss occurs and transition noise appears in the output signal. To deal with this problem, an automatic variable delay line unit (AVDL) was employed after the switching unit to correct the data errors. That unit is also equipped with a function for outputting a video signal with optimum timing in accordance with a synchronization signal. As a result, the missing data can be replaced with the correct data at the point of switching within the vertical blanking interval. Normal video output with video switching operation at arbitrary timing has been confirmed 14). 3.3 17-inch LCD In studios and OB vans, multiple full-featured 8K video must be monitored and adjusted in a small room. For this purpose, we developed a 17-inch 8K LCD that can handle 8K signals at a frame frequency of 120 Hz. The appearance of the display is shown in Fig. 6 and the specifications are listed in Table 4. 15

input (optical) 1 Switching unit AVDL unit Data output (optical) 1 2 3 Crosspoint switch Data Data 2 3 4 Data 4 Synchronization signal Operation Controller Clock generation Figure 5: Configuration of 8K/120 Hz blanking switcher 10 Gbps signal (optical, O) 24 10 Gbps signal (electrical, E) 1 10 Gbps signal (electrical, E) 24 Synchronization/control signal The display has a pixel count of 7640 4320 (pixel density of 510 ppi *3 ) and was fabricated using lowtemperature polysilicon technology *4. The display has Figure 6: 8K/120 Hz 17-inch LCD Table 4: Specifications of 8K/120 Hz 17-inch LCD Panel size 17.3 inches Effective pixel count 7,680 4,320 Pixel structure RGB stripe Pixel density 510 ppi Maximum luminance 500 cd/m 2 Contrast ratio 2,000:1 120/1.001 Hz, 60/1.001 Hz a contrast ratio of 2000:1, a maximum luminance of 500 cd/m 2, and a viewing angle of 160 or more in both the horizontal and vertical directions. The display does not feature a wide color gamut and HDR capability and continued improvement is required to satisfy the required performance for a full-featured 8K production. 3.4 Waveform monitor We have developed a waveform monitor for the full-featured 8K signals that are transmitted over the 15). The monitor can be used to observe the physical performance of 24-channel optical signals at 10.692 Gbps each within the, waveforms and payload ID of 8K video signals, 22.2 multichannel audio signals 16) and time code signals multiplexed as auxiliary data with the video signal 17). The appearance of the monitor is shown in Fig. 7 and the specifications are listed in Table 5. The monitor can measure the received optical power, phase difference, bit error rate, and can also check other characteristics of 10 Gbps optical signals. *3 Pixels per inch *4 A technique for forming polycrystalline silicon on a glass substrate at a low temperature of 500 C, enabling higher performance of the thin-film transistors that drive pixels and higher display resolution. 16

Input Output Size Weight Power consumption Figure 7: 8K waveform monitor Table 5: Specifications of 8K waveform monitor Synchronization signal 4K 3.5 Color grading equipment Pixel count Bit depth Black burst, Tri-level 8K (7,680 4,320) 120/1.001 Hz, 60/1.001 Hz 12, 10 bits same as the input parameters 60/1.001 Hz Sampling YCbCr* 4:2:2 3G-SDI 4 230 (W) 123.8 (H) 350 (D) mm 5.6 kg 200 W (Maximum) * Luminance signal Y and chrominance signals Cb and Cr specified by ARIB STD B56 We developed a color grading equipment *5 that performs video for full-featured 8K signals 18). The appearance of the equipment is shown in Fig. 8 and the specifications are listed in Table 6. The equipment features inputs and outputs and can perform color, scratch, color aberration *6, and contour (detail) for a full-featured 8K video signal in real time. For video tone, the process is performed in the linear (light) signal space so that appropriate process can be performed for HDR signals. A down-conversion function is also equipped to enable monitoring at 4K or 2K resolution. *5 Video color adjustment *6 A shift in image colors caused by differences in optical wavelength created when light passes through a lens Figure 8: 8K/120 Hz color grading equipment Table 6: Specifications of 8K/120 Hz color grading equipment Function Input Outputs Video 1 1, 4K (3G-SDI 4), HDTV (HD-SDI 1) Color, scratches, chromatic aberration, details Resolution (8K, 4K, 2K), Downconversion (120P, 60P, 60i), Color gamut (BT.2020, BT.709) Processing speed (latency) 1 frame or less Size 430 (W) 350 (D) 88 (H) mm (Main unit) 3.6 120 Hz time code A time code *7 is an indispensable technical element for program production and is mainly used in video editing and for playback with synchronization between video and audio equipment such as for audio sweetening (MA) work *8. We developed a high-frame-rate time code (HFR TC) for full-featured 8K video of 120 Hz and contributed to the standardization 19). HFR TC is designed to be compatible with the 30 Hz and 60 Hz frame frequencies that have been used for HDTV. The HFR TC extends the current time code signal by allocating previously unused binary bits to high frame counting, thus adding support for 120 Hz signals. The HFR TC can therefore be handled with equipment for the existing time code up to 60 Hz. Development and standardization have been moving forward for an HFR TC labeling *9, HFR TC multiplexing *7 Time information for video signals in which a unit of time information is associated with each video frame *8 An audio production task in which multiple sounds such as narration, music, and sound effects are synchronized with edited video *9 A method of appending the time code to each video frame 17

onto signals 17), and an interface for the transmission of the HFR TC using HD-SDI. We prepared a prototype HFR TC signal generator/ 20). The appearance of the prototype is shown in Fig. 9. For frame-by-frame labeling of the HFR TC and multiplexing the HFR TC on the for transmission, we developed a HFR TC multiplexer/demultiplexer. The compression recorder described above is also equipped with a function for generating an HFR TC signal. Using this equipment and the system shown in Fig. 10 (a), we verified the compatibility of the HFR TC with the conventional time code. In Fig. 10 (a), the HFR TC generated by the compression recorder is multiplexed on the together with the video signal and then transmitted. The HFR TC multiplexer/demultiplexer then separates a single time code signal over HD-SDI that is received by the HFR TC signal generator/, and the display of the HFR TC is confirmed, as shown in Fig. 10 (b). The two rightmost digits Figure 9: 120 Hz high-frame-rate time code (HFR TC) generator/ on the top display in each photograph in Fig. 10 (b) indicate the frame number read by the conventional time code (0 to 29). The three rightmost digits on the lower display indicate the frame number read by the HFR TC signal generator/ (0 to 119). The frame number displayed on the conventional time code is one-fourth the frame number displayed on the HFR TC, confirming that frame number compatibility is ensured in full-featured 8K production, even when conventional 30 Hz or 60 Hz equipment is included in the system. 4. Construction of a test production system for full-featured 8K We constructed a test production system for full-featured 8K that comprises all of the production equipment described in section 3 being connected with the. The system configuration is illustrated in Fig. 11 and the appearance is shown in Fig. 12. The construction of the testing system took into account compatibility for integration with existing HDTV systems and a smooth transition to 8K systems. In the test production, signal switching was performed by the production switcher for two video signals input from compression recorders playing out prerecorded 8K/120 Hz video. At the same time, the full-featured 8K signal was observed on the waveform monitor and the video image was presented on the 17-inch LCD. The HFR TC generated by the recorder was multiplexed on the and transmitted 120 Hz time code signal generation (internal) 8K/120Hz Data compression recorder (transmission with 120 Hz time code multiplexed) Time code demultiplexer Conventional 30 Hz Time code (8K video signal) HD-SDI (time code signal) 120 Hz Time code Conventional 30 Hz Time code 120 Hz Time code Frame number presents 4 times value (a) Compatibility testing system (b) Display comparison Top display in each photo: conventional 30 Hz time code Lower display in each photo: 120 Hz time code Figure 10: High-frame-rate time code (120 Hz) and conventional time code (30 Hz) 18

8K/120 Hz HDTV 8K connection with 8K blanking video switching 8K signal monitoring 120-Hz time code transmission Backward compatibility Resolution, frame rate, gamut conversion Time code compatibility (30-Hz reading) 120-Hz time code generating 120-Hz time code 30-Hz time code HDTV Display 8K Recorder 8K Recorder 8K blanking switcher demultiplexer 8K LCD Resolution/ conversion Gamut conversion HDTV Recorder 8K Waveform monitor Sync generator (a) Connections of full-featured 8K production system Figure 11: Configuration of 8K/120 Hz production testing system (b) HDTV system Figure 12: Full-featured 8K (120 Hz) test production system via the production switcher, after which it was passed through the HFR TC multiplexer/demultiplexer, and then, the HFR TC signal generator/ successfully displayed the HFR TC. Standard dynamic range video content was used as the signal source in the test. The color grading and down-conversion for HDR video content are planned as future work. The construction of the integrated system that connects the developed full-featured 8K production equipment and the verification test as described above have demonstrated the feasibility of 8K/120 Hz program production including recording, switching and display, and the possibility of maintaining compatibility with the conventional 60 Hz system. The video playback test using the system was demonstrated at the NHK STRL Open House 2016 (Fig. 12). 5. Conclusion We have developed various types of equipment that are required for 8K/120 Hz program production and constructed a prototype of integrated production system for full-featured 8K. The developed compression recorder, simple production switcher, waveform monitor, and color grading equipment and the time code generator/ were connected via an optical transmission interface (). Transmission and switching of full-featured 8K video signals with the 120 Hz time code at 144 Gbps were confirmed. In the current 8K test satellite broadcasting, programs are produced and broadcast at a frame frequency of 60 Hz. For the presentation of sports and other content that involves fast movement, a higher frame frequency is required for less motion blur and smooth motion reproduction. We continue to develop technologies for shooting, production, compression, transmission, and display to implement high-frame frequency 8K production. Furthermore, owing to display technologies that enable presentation of brighter and higher contrast pictures in recent years, HDR is being introduced in broadcasting. An 8K HDR program production system and trial live production were successfully demonstrated at the NHK STRL Open House 2016 (Fig. 13) 21). It is expected that more appealing content is produced by the full-featured 8K with the highest potential of the resolution, color gamut, bit depth, frame frequency, and dynamic range. 19

Figure 13: 8K HDR live production system (60 Hz) With an eye to the Tokyo 2020 Olympic and Paralympic Games and beyond, we will continue to develop the equipment needed to realize full-featured 8K production and postproduction, and proceed with research that will make it possible to provide viewers with enhanced 8K broadcasting services. References 1) Rec. ITU-R BT.2020-1, Parameter Values for Ultra-High Definition Television Systems for Production and International Programme Exchange (2014) 2) SMPTE ST 2036-1-2014, Ultra High Definition Television -Image Parameter Values for Program Production (2014) 3) Association of Radio Industries and Businesses, UHDTV System Parameters for Programme Production, ARIB STD- B56-1.1 (2015) (in Japanese) 4) Rec. ITU-R BT.2100-0, Image Parameter Values for High Dynamic Range Television for Use in Production and International Programme Exchange (2016) 5) K. Masaoka, Y. Nishida, M. Sugawara, E. Nakasu and Y. Nojiri: Sensation of Realness from High-Resolution Images of Real Objects, IEEE Trans. Broadcast., Vol. 59, No. 1, pp. 72-83 (2013) 6) M. Emoto, Y. Kusakabe and M. Sugawara: High-Frame-Rate Motion Picture Quality and Independence of Viewing Distance, J. Disp. Technol., Vol. 10, No. 8, pp. 635-641 (2014) 7) Association of Radio Industries and Businesses, for UHDTV Production Systems, ARIB STD-B58-1.1 (2015) (in Japanese) 8) SMPTE ST 2036-4, Ultra High Definition Television -Multi-link 10Gb/s Signal/Data Using 12-Bit Width Container (2015) 9) Rec. ITU-R BT.2077-1, Real-Time Serial Digital s for UHDTV Signals (2015) 10) K. Kitamura, T. Yasue, T. Soeno and H. Shimamoto: Full- Specification 8K Camera System, NAB Broadcast Engineering Conference Proceedings, pp. 266-271 (2016) 11) R. Funatsu, K. Kitamura, T. Yasue, D. Koide and H. Shimamoto: 8K HDR Cameras Using Hybrid-log Gamma, ITE Tech. Report, Vol. 40, No. 23, BCT2016-59 (2016) (in Japanese) 12) E. Miyashita and T. Kajiyama: Compact Camera Recorder for Super Hi-Vision, IEEE International Conference on Consumer Electronics (ICCE) 2014 Digest, pp. 165-166 (2014) 13) J. Yonai, T. Yamashita and Y. Nishida: A Blanking Switcher for Full-Featured 8K Super Hi-vision, ITE Annual Convention, 22D-2 (2016) (in Japanese) 14) SMPTE RP 168, Definition of Vertical Interval Switching Point for Synchronous Video Switching (2009) 15) T. Soeno, Y. Ikeda and T. Yamashita: 8K Waveform Monitor with Signal Analyzer, ITE Tech. Report, Vol. 40, No. 14, BCT2016-34 (2016) (in Japanese) 16) Association of Radio Industries and Businesses, Audio Data Format in the for UHDTV Production Systems, ARIB STD-B64-1.0 (2015) (in Japanese) 17) Association of Radio Industries and Businesses, Time Code Format in the for UHDTV Production Systems, ARIB STD-B68-1.0 (2015) (in Japanese) 18) T. Hayashida and T. Yamashita: Color Grading Equipment for Full-Featured 8K SHV, ITE Annual Convention, 22D-2 (2016) (in Japanese) 19) SMPTE ST 12-3, Time Code for High Frame Rate Signals and Formatting in the Ancillary Data Space (2016) 20) T. Soeno, N. Shirai, D. Koide and T. Yamashita: 120 Hz Frame Rate Time Code Transmission System for 4K and 8K Program Production, ITE Tech. Report, Vol. 40, No. 14, BCT2016-46 (2016) (in Japanese) 21) D. Koide, T. Yamashita, R. Funatsu, N. Shirai, Y. Ikeda, Y. Nishida and T. Ikeda: 8K UHDTV-HDR Live Program Production System-Construction and Trial, ITE Tech. Report, Vol. 40, No. 23, BCT2016-58 (2016) (in Japanese) 20