Public exhibition information

Transcription

1 NHK(Japan Broadcasting Corporation) Science & Technology Research Laboratories Public exhibition information

2

3

4 Floor Plan 7F Kick-the-8K-Target-Out (Lawn Square) Live Video 1F From 1F Lecture Thu. Research Presentation (Thu.) M 1 Overview of 8K Super Hi-Vision To BF To 7F From BF Auditorium Reception (Sat. Sun.) (Guided Tours) 8K Super Hi-Vision Theatre Lecture Thu. Reseach Presentation Thu. 31 Elevator Washroom OUT IN 1 Hybridcast for 8K Super Hi-Vision 2 Compact 120-fps 8K Super Hi-Vision Camera 3 8K Super Hi-Vision Satellite Broadcasting System 4 Full-specification 8K Super Hi-Vision Production System 5 8K Super Hi-Vision Distribution Technology for Cable TV Networks 8 Flexible Organic Light-emitting Diode Display 9 Integral Three-dimensional Television 10 Spatial Light Modulator Driven by Spin-Transfer Switching 31 8K Super Hi-Vision "Theater Camera" M NHK Broadcast Museum 6 7 Compact 8K Super Hi-Vision Recorder with Removable Memory 8K Super Hi-Vision Sound Reproduction with Display-integrated Loudspeakers

5 Listening room for Exhibition 7 Elevator Washroom BF S S1 From 1F To 1F Lounge 28 J S2 Questionnaire & Lounge H T2 T1 30 P Poster Exhibit 11 Technologies for Advanced Hybridcast 12 TV Program Navigation Based on Estimation of Viewer's Interests 13 Accuracy Improvement on Live Captioning System Automated by Speech Recognition 14 Sign Animation Synthesis System Partially Incorporating Facial Expressions 15 Tactile Presentation Technology to convey 2 & 3D information 16 Transmission Technologies for the Next Generation Digital Terrestrial Broadcasting 17 MMT-based Transport Technology for Advanced Services in 8K Super Hi-Vision 18 Advanced Conditional Access System 19 8K Super Hi-Vision PL Mount Lens & Baseband Processor Unit megapixel Image Sensor 21 Real-time Spatio-temporal Video Format Converter 22 Video Retrieval System Enabling to Search by New Vocabulary 23 Privacy Preserving for Integrated Broadcast-broadband Services 24 Underwater Visible Light Wireless IP Transmission Technology TDD Bidirectional Digital FPU Enabling High-speed 25 File Transfer 26 Video News Gathering over Software Defined Network 27 Professional Digital Wireless Microphone by OFDM System CMOS Image Sensor Overlaid with 28 Photoelectric Conversion Layer 29 Technologies in The Broadcasting Station 30 Utilization and Development of NHK's Technology P Poster Exhibit T1 Haptic TV Touch me if you can T2 Augmented TV S1 Hybridcast Integrated Service by Broadcasting Stations and S2 Internet Service Companies J Digital Broadcasting Reception Consultation Desk H NHK HEART PLAZA (only on 31st May and 1st June)

6 Fusion of Broadcast and Broadband 1 Towards Hybridcast for 8K Super Hi-Vision the launch of advanced Hybridcast services We are studying technology to make Hybridcast even more attractive to users. Here, we show an example of a service that takes advantage of the ultra-high-resolution large screen of 8K Super Hi-Vision (8K SHV) to display a wealth of information associated with TV programs. 8K SHV Hybridcast This demonstrates a multi-window service that displays a wealth of information associated with TV programs that is sent via broadband, in the form of easy-to-see lists. HTML5 1, with its excellent powers of expression and affinity for the Internet, is ideal for various displaying features such as CG animation and the simultaneous presentation of different video feeds via broadcast and broadband. 8K SHV data transmission method Using the MMT 2 media transport method, we can combine various data streams delivered via broadcast and broadband and present them in synchronization. This capability also makes it simple to display program-related content on mobile terminals. 8K SHV closed captioning and text subtitling We propose to use UCS 3 character encoding, which is widely used around the world, and TTML 4 encoding, which can specify control information such as the timing and layout of a display, for closed captioning and text subtitling. Besides the conventional static display at the bottom of the screen, this system can provide an extensive range of data through Hybridcast applications. We are conducting research and development aimed at standardizing the technical specifications and broadcast systems in the expectation of the launch of 8K Super Hi-Vision trial broadcasts in HTML5 (HyperText Markup Language version 5): Web-standard description language that is being standardized by the World Wide Web Consortium (W3C). 2 MMT (MPEG Media Transport): An international-standard media transport method that handles media transmission in heterogeneous network environments. 3 UCS (Universal Multiple-Octet Coded Character Set): An international-standard character encoding that is stipulated in ISO/IEC TTML (Timed Text Markup Language): A markup language that makes it possible to specify the timing and layout of text displays. Basic information Simultaneous delivery of content and control information Broadcast transmission path 8K SHV TV Broadcaster Broadband transmission path Link Service providers Extended information Individual content delivery and bidirectional content Mobile terminal Service example

7 8K Super Hi-Vision 2 Towards Compact 120-fps 8K Super Hi-Vision Camera diverse 8K Super Hi-Vision video shootings We are developing various cameras with the aim of building a 120-fps 8K Super Hi-Vision video production system. We recently developed a compact 120-fps 8K Super Hi-Vision camera that uses a new 33-megapixel 120-fps single-chip CMOS image sensor. 8K Super Hi-Vision video with smooth motion This camera captures video at 120 frames per second twice the rate of current 8K SHV video. This allows it to capture fast-moving action (like sports) with greater clarity. Compact, lightweight camera head The camera head measures cm (W H D) and weighs only 2 kg, and it can be mounted in various ways (e.g., on cranes and remote-controlled platforms), allowing video to be captured from a variety of shooting angles. Optical transmission unit for additional functionality When the camera head is connected to a dedicated optical transmission unit, it can be operated using the same hybrid fiber-optic cable as conventional HDTV cameras. This also enables the addition of other functions such as lens control, intercom, return inputs, and tally inputs. We are working on research and development of 8K Super Hi-Vision video cameras such as full-spec single-chip color cameras that has a full-pixel count, 120-fps frame rate, and a wide color gamut. Our research on CMOS image sensors is being jointly conducted in collaboration with Shizuoka University. Currently only compatible with 60-fps video Camera head Video signal Control signal CCU (signal processing unit) 8K 120-fps video 120-fps display device Width : 15.1 cm Height : 12.5 cm Depth : 13.5 cm Weight : 2 kg ( excluding lens) Processes 8K 120-fps video Control panel Overview of the exhibit

8 8K Super Hi-Vision 3 Towards 8K Super Hi-Vision Satellite Broadcasting System test broadcasting of 8K Super Hi-Vision in 2016 We are researching efficient video coding technology and large-capacity satellite transmission technology in expectation of the launch of 8K Super Hi-Vision (8K SHV) test broadcasting in We are exhibiting a satellite broadcasting system that uses these two technologies. System intended for test broadcasting in 2016 The test broadcasting starting in 2016 will efficiently compress 8K SHV video while maintaining high image quality by using the latest video coding scheme, MPEG-H HEVC/H.265 1, and transmit it in compliance with the ARIB standard, Transmission System for Advanced Wide Band Digital Satellite Broadcasting. Satellite broadcasting transmission system enabling large-capacity transmission By increasing the symbol rate and utilizing 16APSK (7/9) 2, we can transmit a signal of approximately 100 Mbps using a single satellite transponder. 3 Improvement of 8K SHV HEVC encoder The exhibit features our 8K SHV real-time encoder with the latest video coding scheme, HEVC. We have improved the image quality by adjusting the rate control method and added a multiplexing function. We are working on an 8K SHV codec system incorporating the HEVC scheme and MMT 4 and on 8K SHV satellite broadcasting using the 12-GHz band. Part of this research and development is supported by the Ministry of Internal Affairs and Communications, Japan, through its project, Research and development of basic technology encouraging effective utilization of frequency for the next-generation broadcasting. 1 MPEG-H HEVC (High Efficiency Video Coding)/H.265: A video coding scheme which was standardized internationally in a collaboration between ISO/IEC and ITU in APSK (7/9): One of transmission parameters in this system. Using a modulation method that transmits four bits of information simultaneously to make the carrier wave 16-ary amplitude and phase differences, we use 7/9 as the coding ratio of error correction. (APSK: Amplitude and Phase-Shift Keying) 3 The worst-month availability in the Radio Regulations is at least 99.7% at the upper limit (60 dbw) of output power. 4 MMT (MPEG Media Transport): International standard multiplexing method that supports delivery of multimedia content over heterogeneous networks. 12-GHz band broadcasting satellite Transmission of one 8K SHV program by a single satellite transponder 17-GHz band 16 APSK, etc. Occupied bandwidth: 34.5 MHz 12-GHz band Compliance with ARIB standard, Transmission System for Advanced Wide Band Digital Satellite Broadcasting Modulator Demodulator Encoder Transmitter Compliance with HEVC/H.265 Decoder Under development Receiver of 8K SHV test broadcasting

9 8K Super Hi-Vision 4 Towards Full-specification 8K Super Hi-Vision Production System Production of 8K/120Hz/12-bit/Wide-color-gamut Video We have developed a video production system for upper-end 8K Super Hi-Vision (8K SHV) which is called full-specification 8K SHV 1. 8K SHV equipment compatible with 120-Hz frame frequency All of the equipment (camera, video transmission optical interface, and display device) operates at a frame frequency of 120 Hz. Full-specification 8K SHV camera We produced the world s first full-specification 8K SHV camera system utilizing CMOS image sensors for capturing 8K images at 120 Hz and a prism designed for wide-color-gamut colorimetry 2. Optical interface for connecting 8K SHV video devices We have developed a device interface 3 that can transmit a full-specification 8K SHV video signal (approx. 144 Gbit/s) over a single optical cable. We are continuing with our research and development aimed at implementation of a full-specification 8K SHV production system. The research on the image sensor for full-specification 8K SHV is being conducted in collaboration with Shizuoka University. 1 The video parameters of 8K SHV system has been standardized in international standards (Rec. ITU-R BT.2020 and SMPTE ST ) and Japanese standards (ARIB STD-B56). A system with an 8K resolution ( resolution), 120 Hz frame frequency, bit depth of 12 bits, and wide-color-gamut colorimetry is referred to as Full-specification 8K SHV. 2 8K SHV adopts a wide-color-gamut colorimetry that enables the rendering of even more vibrant colors which cannot be representative by conventional television system. 3 Specification of this interface was standardized as ARIB STD-B58 in March Full-specification 8K SHV camera system 8K-resolution, 120-Hz display Full-specification 8K SHV Pixel count (Horizontal Vertical) Frame frequency 7,680 4, Hz (Progressive) Optical transmission interface Transmission of full-specification 8K SHV signal using a single optical cable Bit depth Colorimetry 12 Wide-color-gamut Configuration of full-specification 8K SHV Production System

10 8K Super Hi-Vision 5 Supporting 8K Super Hi-Vision Distribution Technology for Cable TV Networks diverse services for the 8K Super Hi-Vision era We are conducting research aimed at supplying subscribers with 8K Super Hi-Vision (8K SHV) broadcasting via cable TV networks. This exhibit presents a technique that uses multiple channels to deliver 8K SHV signals (which have 16 times as many pixels as ordinary HDTV). Enabling 8K SHV transmissions over existing cable TV facilities By splitting an 8K SHV signal across multiple cable TV channels (64QAM 1 or 256QAM), it can be delivered to subscribers without any changes to the existing transmission facilities. In this exhibit, an 8K SHV signal is transmitted from the head end of a real operating cable TV station via a transmission path to this exhibition hall, where the QAM channels are received and recombined into an 8K SHV program. Flexible support using broadcasting & communications To facilitate the deployment of diverse services such as 8K SHV broadcasting and stored-content services while ensuring compatibility with existing digital broadcasting services, we developed a technique whereby signals formatted as IP packets (which are widely used in communications) can be delivered over cable TV in addition to MPEG-2 TS 2 signals (currently used for digital broadcasting). We have submitted a proposal including this technology to the Japan Cable Television Engineering Association (JCTEA) as an 8K transmission standard. We hope to have a standard in place before the launch of 8K SHV test broadcasting. 1 QAM (quadrature amplitude modulation): A digital modulation scheme where information is conveyed by the phase and amplitude of a carrier wave. 2 MPEG-2 TS (Transport Stream): A transmission signal format used in digital broadcasting. Cable TV station subscriber Subscriber Signals received via satellite 8K SHV Modulation Cable TV transmission path Demodulation 8K SHV Existing broadcasting Split / Multiplex 256QAM 256QAM 64QAM 256QAM 256QAM 64QAM Recombine 8K SHV Existing broadcasting 256QAM 64QAM Frequency Cable TV transmission of 8K SHV on multiple channels

11 8K Super Hi-Vision 6 Highly Compact 8K Super Hi-Vision Recorder with Removable Memory mobile equipment for producing 8K Super Hi-Vision programs We are researching a compact video recorder aimed at bringing a high level of mobility to the production of 8K Super Hi-Vision (8K SHV) programs. Here, we are exhibiting a compact 8K SHV video recorder that uses a removable high-speed, high-capacity solid-state memory pack. A compression and recording method that maintains high picture quality We have made it possible to record pictures with high quality by achieving efficient compression after interpolation processing on the green signals from an 8K SHV camera, which have the greatest effect on picture quality. High-speed, high-capacity solid-state memory pack We have developed an integral removable 8K SHV solid-state memory pack by separating the memory unit and recording control unit. We have also implemented high-speed recording based on a recording control technique that reduces fluctuations in write speed and an efficient write technique whereby data is recorded in a series of large blocks. This makes it possible to record at up to 12 Gbps and store approximately 45 minutes of 8K SHV content. We will continue to improve the compression efficiency and convenience of this technology with a view to creating a practical 8K SHV compact video recorder. We will also work at increasing the speed and capacity of the solid-state memory pack. This compact video recorder is being conducted in collaboration with Tokyo Electron Device, Ltd. Solid state memory pack Our new compression and recording method Frame1 Frame2 Frame3 Frame4 Camera output Data interpolation Image compression Record as a sequence of frames Compact video recorder using a solid-state memory pack

12 8K Super Hi-Vision 7 Enabling 8K Super Hi-Vision Sound Reproduction with Display-integrated Loudspeakers viewers to enjoy 3D sound at home 22.2 multichannel sound is optimally reproduced using speakers arranged around the listener. To reproduce 22.2 multichannel sound in a typical living room that does not have space to install numerous speakers around the listening point, we are developing technology that mounts all of the speakers around the edge of the display. Audio signal processing that implements 3D sound Sound will reach the left and right ears having differences in arrival time, level, and timbre that depend on the source s direction and distance, and people are able to perceive the direction of a sound source from those differences. We use binaural signal processing so that the loudspeakers mounted around the edge of the display can express sound as if it were coming from all directions. Better sound image localization using numerous loudspeakers We have improved the localization of the sound image in a way that accommodates slight shifts in the positions of the listeners by using a numerous loudspeakers and enhancing the independence of their sound propagation paths. We will continue our work to improve the sound image localization and expand the listening area with good localization characteristics while maintaining sound quality. Part of this research was being conducted in collaboration with Foster Electrical Company, Ltd. Sound image localization: Perceiving the sound image at a position determined by direction and distance. The sensation of stably perceiving the sound image at a certain position. Loudspeakers Viewing 8K Super Hi-Vision at home

13 Next Generation Broadcasting Devices 8 In Flexible Organic Light-Emitting Diode Display pursuit of a lightweight, large-scale display for 8K Super Hi-Vision We are developing a light and super-slim flexible display that is easy to carry around. Some of our elemental technologies aimed at developing large displays for 8K Super Hi-Vision are on exhibit. One example is the inverted OLED, which is expected to extend the life-time of the flexible display. Inverted OLED, a potential way of making a long-life diode In an effort to extend the lifetime of OLEDs, we have developed a new electron injection layer without using materials that react with oxygen and moisture. The anode and cathode of the OLED was inverted so that this new material could be layered to form a film, thus called an inverted OLED. Printing as a means to fabricate larger screens On exhibit is a display with print-fabricated wiring and TFTs (thin film transistors), which are easier to incorporate into larger screens. This is another example of our elemental technologies for large, sheet-type displays. We will continue to improve OLEDs to create displays with lower power consumption and longer life and will continue to research new technologies for display production and TFT development. The research on inverted OLEDs is being conducted in collaboration with Nippon Shokubai Co., LTD. Conventional structure Conventional Electron Injection Material (Susceptible to oxygen and moisture) Sealant Inverted OLED Standard OLED Electric current Cathode Anode Anode Cathode Electric current New Electron Injection Material (Unsusceptible to oxygen and moisture) Light Emission Plastic Substrates Display Surface Cross section diagram of flexible OLED display

14 Three-dimensional Television 9 Making Integral Three-dimensional Television 3D images more natural and easier to view We are researching integral three-dimensional television with the aim of building a 3D television that can be viewed without 3D glasses. This exhibit shows how it is possible to capture 3D images of any object and how high-quality 3D images can be captured and displayed. Generation of 3D images from an infrared/color camera array We are researching a technique for generating integral three-dimensional images from multi-view video images. This technique previously suffered from impaired picture quality when the feature points are difficult to find, e.g., when the visual object is plain and textureless. We have found a way to overcome this problem by illuminating objects with an infrared dot pattern in addition to ordinary lighting. By capturing images with a camera array comprising infrared and color cameras, we can expand the range of 3D imaging possibilities and generate integral 3D images of diverse visual objects, including ones with no visible textures. Integral 3D imaging equipment using multiple cameras To capture high-quality 3D images, we are developing integral 3D imaging equipment consisting of a lens array and multiple cameras. We have incorporated seven cameras into this equipment. This allows us to capture 3D video images over a range approximately 2.5 times wider and taller than what can be captured with a single camera. Integral 3D display device using multiple displays A large number of pixels are needed to display high-quality integral 3D images, but there are limits to what can be achieved with a single display. We have therefore developed a display device that uses a larger number of pixels to regenerate 3D images of higher quality by using multiple displays to combine images. We will continue with our research and development on imaging and display technology for creating high-quality threedimensional images. Integral three-dimensional television: Television that reproduces three-dimensional images by capturing and displaying pictures with a lens array consisting of micro lens elements. Generate 3D image from infrared/color camera array Color camera Color image Integral 3D image generated from multi-view color and infrared images Display device using multiple displays Magnifying optical system Infrared dot projector Dot pattern (feature points) Infrared image Infrared camera textureless objects over a wide range 3D image Capture equipment using multiple cameras Lens array Cameras over a wide range Lens array Object Four displays Image produced by combining four pictures Core technologies of integral 3D television

15 Three-dimensional Television 10 Technology Spatial Light Modulator Driven by Spin-Transfer Switching for the future holographic 3D televisions We have been studying ultra-high-density spatial light modulators 1 (SLMs) with very large numbers of pixels, as crucial technology for holographic 3D televisions. Recently, we developed a new spintransfer switching SLM driven with an active matrix method 2. This achievement opens the way to building a large-scale SLM with many pixels. Spatial light modulator driven by an active matrix A spatial light modulator for the display of 3D holographic images requires a high-speed response and ultra-high resolution. By using an active matrix driving method with a transistor incorporated into each pixel, we recently developed the first spatial light modulator with spin-transfer switching, which can be extended to a very large number of pixels with a high-speed switching response. Light modulator elements using the tunnel magnetoresistance effect A conventional spatial light modulator draws a large electrical current. As a way of reducing this current, we have developed light modulators using tunnel magnetoresistance 3. Each pixel is operated by spin transfer magnetization reversal, whereby the direction of the current flowing through the pixel controls the magnetization direction of the light modulator. To implement an ultra-high-resolution spatial light modulator, we will develop a nano-scale fabrication process and improve the device performance. This research was partly supported commissioned by the National Institute of Information and Communications Technology (NICT), Japan as part of the project, Research and development of ultra-realistic communication technology through innovative threedimensional image technology. We also collaborated with Nagaoka University of Technology. 1 Spatial light modulator: A device consisting of a 2D arrangement of tiny optical elements (equivalent to pixels) that can control the state of light (amplitude, phase, etc). 2 Active matrix method: A method for individually driving selected pixels by means of transistor switching elements at each pixel. 3 Tunnel magnetoresistance: A phenomenon whereby the electrical resistance of an insulating film sandwiched between two magnetic films changes depending on the polarity of a tunneling current flowing across the insulating film. Spatial light modulator Incident light Function of the spatial light modulator Interference fringes 3D image Basic principle of displaying holographic 3D images with a spatial light modulator

16 Fusion of Broadcast and Broadband 11 Toward Technologies for Advanced Hybridcast the enhancement of integrated broadcast-broadband services We have conducted research and development for enhancing Hybridcast technologies to realize more advanced services. This exhibit presents four new elemental technologies which provide a new kind of TV experience with a diversity of applications. Support for VOD and recorder function A VOD service can be launched through a program guide application provided by the broadcaster. This functionality also allows to launch applications with TV programs even when watching the TV programs on the recorder or over VOD. Non-broadcast-oriented managed applications This type of application allows third party service providers other than broadcasters to provide various cross-channel services. "Push" delivery technology "Push" delivery technology, which carries various information over the Internet, enables to show a wide variety of information on Hybridcast receivers and mobile terminals in real time. Technology for management and distribution of program-related data Supplying third party service providers with diverse information associated with broadcast programs (metadata, etc.) enables them to develop program-related applications. We continue contributing to standardization of these elemental technologies and enhancing Hybridcast services. Broadcast Broadcast-oriented managed application (Managed by broadcaster) Support for VOD and recorder function Push notification Broadcaster Application Management and distribution of program-related data Metadata, etc. "Push" delivery technology VOD Push notification Tablet Hybridcast receiver Push notification Smartphone Application Push notification Application market The internet Non-broadcast-oriented managed application Third party service providers Technical elements for advanced Hybridcast

17 Human-friendly Broadcasting Service 12 Help TV Program Navigation Based on Estimation of Viewer s Interests for finding your favorite TV programs We have developed a TV program navigation system that estimates a TV viewer s interests on the basis of his/her behaviors while watching TV and assists the viewer in searching for TV programs that appeal to the viewer taking account of these estimated interests. Technology for estimating the viewer s interests This technology automatically estimates the viewer s level of interest in a program from his/her natural behaviors. Details about his/her interests are then acquired by analyzing the closed captions of the TV programs and the results of the analysis are presented on a tablet terminal as keywords. Help in searching for TV programs We have developed a navigation system that searches for TV programs of interest to the viewer. This system can find the related TV programs based on semantic relations between TV programs by analyzing text information of the TV programs and the Web. We will improve the method of estimating the TV viewer's interests so that we can better customize the TV viewing experience and develop the management system of a huge volume of TV programs linked by the semantic relations. Part of the research into analyzing TV viewer behavior is being conducted in collaboration with the University of Tokyo. Little viewer interest Great viewer interest Extraction of keywords indicating the viewer s interest Keyword selection List of viewer's interests Sesame oil Tempura Deep-fried oysters Prawns Traditional Japanese-style cuisine Uses keywords as departure points of the navigation TV program navigation Analysis of relationships between viewer's behavior and TV program details Tameshite Gatten! Spring vegetables Tokyo Bitter gourd Heartburn Kyoto Okinawa Kyou no Ryori Kyou no Ryori Suiensaa Kyou no Ryori Kyou no Ryori Ingredients Causality Specialties Tempura Large collection of TV programs Surrounding information Automatically generated TV program map Overview of TV program navigation, starting from viewer s interest keywords

18 Human-friendly Broadcasting Service 13 Towards Accuracy Improvement on Live Captioning System Automated by Speech Recognition the Expansion of Live-Captioned Broadcasting We are conducting research and development into speech recognition, with the aim of expanding live-captioned broadcasting to enable a wider range of people to enjoy TV programs, especially the hard of hearing. We are implementing new technology that can generate efficient live captioning using speech recognition, to ensure that even a regional broadcasting station can operate a live-captioning production system. Live-captioning technology for local TV programs With the live-captioning production systems based on speech recognition that are currently in operation, an operator corrects any recognition errors. By deducing corresponding manuscripts and automatically correcting recognition errors, this system makes the operator s work unnecessary and enables the introduction of a live-captioning production system to even a regional broadcasting station. Topic deduction technology for TV variety show To ensure accurate captioning of TV variety shows that cover various different topics, the recognition system must be trained by phrases that are often used within those topics. This can save some of the manual work by deducing the topic from continuity text and automatically selecting texts matched to the topic from a large-scale database. We will continue to work on a system that can improve on the speech recognition capabilities and reduce manual labor of current systems, to ensure that highly accurate live captioning can be applied to more TV programs. This research is a collaborative exhibition with the engineering administration department. With a program such as an informational program or sportscast in which there is a number of performers or a lot of background noise, we recognize an audio feed in which a speaker who is separate from the program s performers repeats the words. Error correction by operator Conventional method TV program audio Manual selection of training texts relating to the topic of the program. Conventional method Recognition results with a few percent of errors remaining Speech recognition According to the Weather Bureau... Average year... News manuscript database Proposed method Live-captioned broadcast It will be fine again today. Proposed method Model of word sequences matched to the topic (dictionary) Recognition results The rainfall in the Kanto district is 49% of the nearly average... Misrecognized word Live captioning character string The rainfall in the Kanto district is 49% of the yearly average... Automatic error correction Automatic selection of training texts from numerous database. Automation to make speech recognition results highly accurate

19 Human-friendly Broadcasting Service 14 For Sign Animation Synthesis System Partially Incorporating Facial Expressions expanded sign language services We are researching technology for translating Japanese into sign language and synthesizing signing animation by means of computer graphics (CGs). The exhibited translation system infers words to be expressed by facial gestures, which play an important linguistic role in sign language, and it composes facial signs and manual signs in CG synthesis. Translation technology for automatically inferring words to be turned into facial expressions In sign language, some words are conveyed by facial expressions. We have developed a CG sign language translation system that infers these words and conveys them via facial expressions simultaneously with the manual signing. CG technology for conveying a variety of facial expressions An extension to TVML has made it possible to use CGs to convey words through facial expressions. Our CG sign language dictionary is now available at NHK Online To help make our signed broadcasts easier to understand and useful to greater numbers of viewers, we are interested in hearing people's opinions of this CG sign language technology. ( We will continue to develop sign language translation technology that can convert a wider range of words through facial expressions and CG sign language generation technology that can produce more richly varied facial expressions. Part of research is being conducted in collaboration with Kogakuin University. TVML (TV program Making Language): A CG-based markup language for TV programs, developed by NHK Science & Technology Research Laboratories (STRL) Japanese "Severe" "Thunderstorm" Infer words to be turned into facial gestures Facial gesture Manual gesture C G High waves and severe thunderstorms... Extend TVML to express various facial gestures Compose facial and manual gestures "Severe thunderstorm" Sign language animation incorporating facial expressions

20 Human-friendly Broadcasting Service 15 Towards Tactile Presentation Technology to convey 2D & 3D information the introduction of Tactile TV We are researching technology that will convey the shape and hardness of 3D objects, such as works of art, together with diagrams and graphs, in a tactile, or haptic, way to people with impaired vision. The haptic presentation system on display enables people to feel the shape and hardness of a depicted object with a sensation that is close to the real thing, and the tactile presentation system can rapidly convey diagrammatic information. Haptic presentation system that reproduces the shape and hardness of a 3D object We have developed a haptic presentation system that uses ultrasound and lasers to enable rapid measurement of the shape and hardness of an object in a contactless manner. It virtually reproduces the shape and hardness of the depicted object on the basis of the measured data. Our previous developments have enabled people to feel the characteristics of a corner or edge of a depicted object by stimulating a number of points on one finger, something difficult to achieve with a single point of stimulus, and to perceive differences in hardness in an easy-to-understand manner. Tactile and powered mechanical leading composite presentation that conveys the content of diagrams and graphs We have developed a method that enables the perception of important locations within a diagram or weather map by using different vibrations to stimulate different locations on a tactile display, and a method of conveying information such as the overall configuration or track in a graph by mechanically leading a finger on the tactile display before the palm or finger obtains the information by touch. This ensures that even unfamiliar content can be conveyed rapidly and reliably to the user. We are continuing to develop our easy-to-understand tactile presentation method and devices with the aim of making information barrier-free. Part of this research is conducted in collaboration with the University of Tokyo. Using Touch to Convey Information that is Difficult to Convey in Words Gives anyone a sensation close to the real thing Faster and more reliable for people with impaired vision Measures shape and hardness in a contactless manner Shape/hardness sensing device High Low Low Transmission of shape and hardness of 3D object from broadcast station High Stimulation by a number of points on one finger Multipoint-stimulus type of haptic presentation device Tactile display Conveys important locations by differences in vibration Powered mechanical leading presentation device Leads finger before it touches content Concept of tactile presentation that conveys a solid object or diagram

21 8K Super Hi-Vision 16 For Transmission Technologies for the Next Generation Digital Terrestrial Broadcasting large-capacity transmissions We are conducting research into large-capacity transmission technologies for the next generation of terrestrial broadcasting, such as 8K Super Hi-Vision (8K SHV). We are conducting long-distance transmission tests of 8K SHV in Hitoyoshi City, Kumamoto Prefecture, and have developed technology that enables mobile reception of a number of Hi-Vision TV programs on one TV channel. Transmission technologies for fixed reception Technologies such as dual-polarized MIMO 1 and ultra-multilevel OFDM 2 enable 8K SHV to be transmitted on a single UHF-band channel. A compressed 8K SHV signal was transmitted from a test station at NHK s Hitoyoshi TV relay station in Hitoyoshi City, Kumamoto Prefecture. This test confirmed that 8K SHV signals could be successfully received at a station 27 km away. Transmission technologies for mobile reception We have confirmed that three Hi-Vision programs (compressed with H ) transmitted on one TV channel can be received, even in a vehicle moving through a constantly changing reception environment, through the application of dual polarized MIMO technology and powerful error-correcting code. We are continuing our research into transmission technologies that will enable simultaneous reception of a single channel of 8K SHV broadcasting for fixed reception and Hi-Vision broadcasting for mobile reception, in a similar manner to current terrestrial digital broadcasting. Part of this research is being performed under the auspices of Research and Development of Basic Technology Encouraging Effective Utilization of Frequency for the Next Generation Broadcasting System program and it founded by the Ministry of Internal Affairs and Communications, Japan. 1 MIMO: Multiple-Input Multiple-Output 2 OFDM: Orthogonal Frequency Division Multiplexing 3 H.264: A method of efficiently compressing video data Mobile reception Large-capacity services such as 8K SHV, intended for fixed reception Robust reception of Hi-Vision programs in moving vehicles and portable terminals Handheld reception of next-generation terrestrial broadcasting

22 8K Super Hi-Vision 17 Towards MMT-based Transport Technology for Advanced Services in 8K Super Hi-Vision Harmonization of Broadcasting and Broadband by MMT MPEG-H MMT 1 is a new media transport method that makes it possible to deliver content by using both broadcasting and broadband. This exhibit introduces MMT equipment including a transmitter and receiver and an example application for 8K Super Hi-Vision broadcasting. Common delivery mechanism on broadcasting and broadband achieved by MMT It is possible to present various types of content on different channels since MMT provides a common delivery mechanism on broadcasting and broadband. Ability to arrange the position of content in the transport layer 2 It is easy to use various devices such as TVs and tablet computers to present content since MMT designates the device and position for displaying each piece of content. Content delivery taking advantage of different delivery channels Media components such as video, audio, etc., that are elements of a TV program can be delivered in a multiplexed form as well as in a separate form depending on the characteristics of the delivery channels. A client can present them in synchronization with each other when they are needed. We will continue developments aimed at advanced services for 8K Super Hi-Vision broadcasting, including building an MMT processor for HEVC real-time encoders and decoders and MMT-based multiplexing devices. 1 MMT (MPEG Media Transport): A media transport method that handles media delivery for multimedia services over heterogeneous network environments, which was standardized by MPEG in Transport layer: A layer that provides media synchronization and multiplexing functions of video, audio, and other signals for their delivery and consumption. Delivery of high-quality video, audio, etc., simultaneously to a lot of users by broadcast Synchronization of video and audio from different channels in the same screen Synchronization of video and audio presented in different devices such as TV and tablets. Broadcast station Delivery of video, audio, etc., individually to devices by broadband Presentation of video, audio, etc., in a synchronized manner even if they are delivered on different channels Presentation of information from broadcasting and broadband in a coordinated manner

23 8K Super Hi-Vision 18 Rights Advanced Conditional Access System protection and conditional access for broadcasting We are researching the next generation of CAS 1 technology that provides rights protection and conditional access to content. We have developed an advanced CAS that is highly secure and enables software to be updated to maintain and improve security. Scrambling system with high security We have developed a scrambling system that supports MMT 2 as a multiplexing system for 8K Super Hi-Vision (8K SHV) broadcasting. It uses a block cipher with a key length of 128 bits (such as AES 3 ) and is capable of switching the encryption algorithm when a vulnerability is detected in the scrambling system. Maintenance and improvement of security We are developing a downloadable CAS that can be updated by making secure downloads to the receiver through broadcasting and communication links, with the aim of maintaining and improving the security of CAS. The content rights protection systems (RMP 4 ) and conditional access systems operating on the downloadable CAS protect 8K SHV programs. We will standardize this as a conditional access system for 8K Super Hi-Vision broadcasting and put it to practical use as the next generation of CAS technology. 1 CAS: Conditional Access System 2 MMT: MPEG Media Transport 3 AES: Advanced Encryption Standard 4 RMP: Rights Management and Protection Advanced scrambling (supporting MMT, etc.) TV receiver Broadcast Communications network Content Key distribution Broadcasting station CAS program Download of CAS functions CAS program (for rights protection) CAS program (for conditional access) Downloadable CAS Advanced CAS overview

24 8K Super Hi-Vision 19 Innovative 8K Super Hi-Vision PL Mount Lens & Baseband Processor Unit Equipment for Promotion of 8K Super Hi-Vision On exhibit are a baseband processor unit that can be used with a high-resolution cinema camera to simultaneously transmit 8K, 4K, and 2K videos, and a single-chip camera lens for 8K Super Hi-Vision. High-resolution 8K Super Hi-Vision video processing using a cinema camera [Baseband Processor Unit] The unit takes raw data from a Sony F65 camera equipped with 8K image sensors and converts them into high-resolution 8K video signals. The signals are automatically down-converted to 4K and 2K so signals in three different qualities can be transmitted simultaneously. Multi-format video production [Baseband Processor Unit] The unit is equipped with an SR memory card slot so you can play out the recorded raw data. It can also up-convert or down-convert 4K signals to 8K or 2K. 8K Super Hi-Vision single-chip camera lens The camera lens is a PL mount that is used with consumer cameras. These technologies will be used for live sports coverage and content production to promote 8K Super Hi-Vision. The baseband processor unit was developed as part of research commissioned by the NexTV-F (Next Generation Television & Broadcasting Promotion Forum) for the "Development and Formation of 8K Content Production Environment". The baseband processor unit was developed in cooperation with Sony Corporation. Single-chip camera lens was developed in cooperation with Canon Marketing Japan, Inc. 1 RAW data: unprocessed data from an imaging device 2 DG: Dual Green. A method of producing 8K resolution by doubling the bandwidth for green which has higher luminous factors than red and blue. 3 3G-SDI: a video transmission standard 4 FIR Filter: Finite Impulse Response Filter, a type of digital filter This exhibitis presented by the Engineering Administration Department. Cinema Camera Head Optical Composite Cable Newly Developed Baseband Processor Unit RAW data 1 Signal Processor 8K DG signal 2 4K Signal SR Memory Card 4K Signal Signal Processor FIR Filter 4 Down Converter 4K signal 2K signal 8K DG signal 2 3G-SDI 3 4 4K signal Baseband Processor Unit

25 8K Super Hi-Vision 20 Towards 133-megapixel Image Sensor a compact, full-resolution 8K Super Hi-Vision camera We are conducting research with the aim of making full-resolution 8K Super Hi-Vision (8K SHV) 1 cameras more compact. We have developed a 133-megapixel image sensor that enables a single chip to capture full-resolution 8K Super Hi-Vision video. Sensor for compact full-resolution 8K SHV camera Until now, it was difficult to reduce the size of the full-resolution SHV camera because it needed three image sensors (i.e.,one for each RGB-color channel) and a color separation prism. The new image sensor can capture full-resolution video with a frame frequency of 60 Hz by itself and be incorporated in a compact camera without a prism. 35mm full-frame, large optical format 2 To accommodate more pixels within a given area of the image sensor, each pixel must be made smaller, and its sensitivity decreases as a result. We have suppressed this decrease in sensitivity by using optical format larger than that of conventional sensors. We have also designed the sensor to have an active area with a diagonal length that is equivalent to that of a 35mm full-frame image sensor, thereby making it possible to use a wide variety of commercially available lenses. The world's largest pixel count for video shooting This image sensor has the largest pixel count of any image sensor currently designed for shooting video. We will incorporate this image sensor into compact full-resolution 8K Super Hi-Vision camera for practical use. 1 Full-resolution 8K Super Hi-Vision: A Super Hi-Vision video signal format having 33 megapixels (7680 horizontal 4320 vertical pixels) for each R/G/B color signal. 2 35mm full-frame optical format: Optical size of image sensor that is equivalent to 35mm film (24 mm x 36mm) (diagonal: 43.27mm). 43.2mm Imaging active area 8640 pixels ( pixels) G R G R B G pixels ( pixels) B G G R G R B G B G Composing each pixel of the video signal from four pixels on the image sensor 133-megapixel image sensor

26 8K Super Hi-Vision 21 Towards Real-Time Spatio-temporal Video Format Converter ultra-high-efficiency video transmissions with spatio-temporal scalability We are conducting research into an ultra-high-efficiency video transmission system that can deliver video signals with various spatio-temporal resolutions 1 ranging from 8K Super Hi-Vision to 2K Hi- Vision. We are demonstrating a real-time video codec system that uses a spatio-temporal video format converter we have developed. Ultra-high-efficiency video transmission system Blocking artifacts can be suppressed by reducing video resolution and frame rate to 2K 60P from videos with higher resolutions, including 4K 120P, before encoding on the transmission side. On the reception side, the decoded video frames are up-sampled to the display resolution by applying spatio-temporal hybrid image reconstruction technology. Resolution conversion using spatio-temporal hybrid image reconstruction technology The spatio-temporal hybrid image reconstruction technology is capable of constructing images with a higher resolution and a higher frame rate than the input video by combining super-resolution technology 2 and frame interpolation technology 3. Adaptive reconstruction using auxiliary side data The reconstruction process on the reception side is adaptively controlled with auxiliary information obtained through parameter optimization on the transmission side. We will develop devices that can process videos up to the level of 8K Super Hi-Vision. We will also introduce new reduction and reconstruction axes other than spatio-temporal axes, such as bit-depth conversion, and techniques for making further improvements to images in order to achieve even higher compression rates. 1 Spatio-temporal resolution: The combination of the frame rate (temporal resolution) and pixel count (spatial resolution). 2 Super-resolution technology: A technique for increasing resolution by supplementing edges and fine patterns. 3 Frame interpolation technology: A technique of compositing an image of an intermediate frame from the frames before and after it. Transmission Transfer Reception 4K 120P 4K 60P 2K 120P Spatio-temporal video format converter (reduction, optimization ) 2K 60P Low-resolution video Local decoded video Side data Optimal selection Synchronization data Video encoding Side data encoder Video stream Side stream Video decoding Approx. 1 to 10% of video stream Side data decoder Spatio-temporal video format converter (reconstruction) Synchronization of received video and side data Optimal restoration based on side data 4K 120P 4K 60P 2K 120P Reconstructive video coding system with real-time spatio-temporal video format converter

27 Enhancing Production 22 For Video Retrieval System Enabling to Search by New Vocabulary effective use of vast amounts of video resources We are conducting research into video retrieval technology enabling users to effectively find desired scenes amidst a huge volume of video material. Traditional systems retrieved target scenes by comparing keyword queries with metadata that was manually attached to the video beforehand. In contrast, we have developed a new retrieval system that can assign new metadata to video footage automatically, enabling users to search for scenes with new keywords that are not in the original metadata. Automatic analysis of what is shown in the video The new retrieval system uses object recognition technology to identify video subjects and similar retrieval technology for video footage. Support of new search keywords by machine learning In object recognition, the system uses training data such as Internet images to learn the visual features of unrecognized objects. The system scans archived video data and determines automatically whether learned objects are present in the video footage. This processing increases the number of searchable keywords beyond those that were manually assigned by the TV producer. Retrieval using images of scenes that are difficult to express in words In similar video retrieval, the user inputs an image as a search query that is close to the desired video footage, and the system retrieves scenes in which the color, texture, and composition are similar to the input. We will continue with our research on advanced retrieval based on video content and work toward deployment of a practical system in cooperation with related departments. This research is conducted in cooperation with IBM Japan, Ltd. User External information Background processing Video retrieval system Yesterday No search results found Image data Mount Fuji Collecting training data for machine learning Object recognition technology Recognized Archived video data Newly recognized Mount Fuji Text data Retrieval history, Internet Mount Fuji Female diver Processing unrecognized objects Mount Fuji Learning visual features Sea, sky Meadow, sky Sky Automobile Mount Fuji Retrieval by keyword Meiko Female diver Japanese food Mount Fuji Japanese food Searchable keywords are gradually increasing. Similar video retrieval Comparison in block units Retrieval by image I want a scene like this Video administrator Video retrieval using object recognition and similar video retrieval

28 Enhancing Production 23 Towards Privacy Preserving for Integrated Broadcast-broadband Services the implementation of convenient and secure services Although integrated broadcast-broadband services can be enhanced in many ways by using personal information, the privacy of TV viewers must also be preserved. That is why we are developing technology that will ensure both the convenience and security of such services. Using a cloud to increase convenience and expand services With services that recommend TV programs or applications individually, the TV viewer must encrypt his/her viewing history and transmit it to each provider in different ways, which means the viewer is inconvenienced by having to do this work. To ease the burden, we have developed a system whereby the viewing history is encrypted only once and saved in a cloud, and the viewer permits specific service providers to acquire the viewing history. We expect that this system will make it easier for businesses to develop services that use huge numbers of viewing histories. Using the provider's attributes to determine which provider can get personal information Privacy can be preserved without any personal information getting leaked, since the viewing history is encrypted and can only be decrypted by providers that have attributes satisfying the disclosure conditions that the TV viewer specifies. Implementation of our scheme in HTML5 1 applications The encryption algorithm is written in JavaScript 2, which runs on HTML5. This enables a viewing history to be encrypted within a practicable time, even in HTML5 applications for integrated broadcast-broadband services. We will expand the functions of the encryption scheme so that it can be applied to other services using personal information, and we will continue research on implementation of large-scale databases and key management servers that will be needed for a practical service. This research is being conducted in collaboration with the National Institute of Advanced Industrial Science and Technology. 1 HTML5: The latest version of the HTML language for creating Web pages 2 JavaScript: A simple programming language suitable for Web browsers We want to obtain as much viewing history as possible. Cloud server TV programs It s OK to hand over my viewing history, but only to providers I can trust. Encryption Broadcaster Encrypted viewing history TV viewer Service provider A Recommended information (TV programs or applications) Service provider B Safe and secure personalized service using viewing history

29 Enhancing Production 24 For Underwater Visible Light Wireless IP Transmission Technology underwater live broadcasts We are researching visible-light wireless IP transmission technology to make it possible to conduct underwater live broadcasts. This exhibit introduces underwater IP transmission technology that can transmit video pictures using visible light, without them being affected by attenuation or suspended particles as the light travels through the water. Wireless IP technology for stable video transmission We have developed a new wireless IP technique for stable video transmission that uses two-way IP transmission technology to send back information about the video signal reception conditions to the transmitter. Adaptive transmission rate control The transmission rates of the video data and error correction data are controlled so that if the sent data is lost due to temporary blocking of the underwater transmission path, the missing parts can be retransmitted with priority. Underwater wireless transmission with a blue LED Radio waves are strongly attenuated in water, but visible light is attenuated less. Hence, we decided to use blue LEDs as the light source. We will perform transmission tests in diverse underwater environments such as swimming pools and sea water, and we will continue with R&D to enable the use of this technology for live broadcasts, including improving the stability and range of the video transmission. We are working with NHK's Broadcast Engineering Department to develop the transmission equipment. Land Underwater camera Stabilized video transmission by IP technology Underwater Video monitor Receiver Transmitter Underwater wireless transmission using blue visible light Illustration of underwater live broadcasts

30 Enhancing Production 25 For TDD Bidirectional Digital FPU Enabling High-speed File Transfer high-speed, reliable transmissions through bidirectional communication Wireless links called FPU are used to send video materials from a relay site to a broadcasting station. The conventional FPU uses simplex communication. We are developing a novel FPU that is capable of duplex communication. This makes it possible to introduce new functions including transfer of filebased video materials and support for a return link to send broadcast video back to the relay site. Implementing two-way transmission using single channel Using a time-division duplex (TDD) scheme to switch efficiently between transmitting and receiving, we implemented bidirectional communication using the same channel as conventional equipment. Modulation and retransmission scheme for improved throughput & efficiency With bidirectional communication, it is possible to automatically select a modulation method adapted to the propagation path and to retransmit packets that are lost in transmission. We exploited this feature by introducing an efficient retransmission scheme and a multi-level modulation method that can switch between different modulation schemes up to 256QAM. Compatible with TCP/IP for greater convenience in relaying video and transmitting video contributions for production The bidirectional FPU is compatible with the TCP/IP protocol suite. This makes the bidirectional FPU uniquely able to support activities such as uploading video files directly from the filming location to the broadcasting station's video contributions server, and operating remote cameras from inside the broadcasting station. We are conducting field transmission tests to verify the transmission performance, and we are working at achieving further speed increases. FPU (Field Pickup Unit): A portable wireless transmission device for the transmission of video materials for TV programs. They are widely used in outside broadcast and for transmitting video materials for program production. High-speed transfer of video materials Error notification, retransmission Transmission of live video Video material server Transmission of broadcast video Relay site Studio Example of functions that can be implemented with a bidirectional FPU system

31 Enhancing Production 26 Supporting Video News Gathering over Software Defined Network news coverage during disasters and emergencies This research focuses on developing news gathering networks for broadcasters to gather footage from various sites and sources and to contribute to quick and agile news production. This exhibit features a technology which takes advantage of how IP networks are shared by users. The system enables users to monitor network traffic and control transmission rates depending on the importance of the footage. Visualizing network traffic and controlling transmission rate on an SDN 1 By visualizing network traffic, editors in newsrooms can choose and prioritize certain feeds over others. This means editors can take into account factors such as how well the news gathering is going or how far into the program the current news program is. Transmission rates can be controlled using a fader-type apparatus. Live transmission: dynamic control of compression rates A glitch or loss during a live feed from the field is inadmissible. With this system, one can send a number of live feeds at once without occupying as much network capacity. This is made possible by dynamically managing video compression rates; it sends high-bit-rate, high-resolution video while live and lower bit-rate video during stand-by. File transfer: prioritizing by controlling transmission rates When sending recorded video files from the field to the newsroom, transmission rates will depend on the priority level determined at the receiving end. The system is compatible with TCP 2 /IP transmission protocol which means there is no need to change your file sharing software in order to use this system. We review how effective the system is in actual news gathering operations and continue our research to improve news production to provide reliable and reassuring information to the public. 1 SDN: Software Defined Network which allows dynamic changes to its configuration 2 TCP: Transmission Control Protocol is a common set of rules that enables transfer of data without errors. Weather-Cam A Live Transmission Live On-Site B is going on air! Change to high-res! IP News Gathering Network Bandwidth Management, Controller Unit Live video On Air News Output Live On-Site B Edited Video News Gathering Site C News Gathering Site D File Transfer of Recorded Material Editors determine priority IP News Gathering Network Editing We need the video file from Site D first! Waiting to edit!

32 Enhancing Production 27 Wireless Professional Digital Wireless Microphone by OFDM System microphone with high sound quality and low latency in the 1.2-GHz band Wireless microphones that are used in the production of broadcast TV programs will have to be shifted to new frequency bands to conform with Japan s frequency reorganization policy. To make this transition smooth, we have developed a new digital wireless microphone that operates in the 1.2-GHz band and provides stable and high-quality audio. Linear PCM audio transmission The use of an OFDM digital system with a frequency bandwidth of 600 khz enables the microphone to transmit an uncompressed linear PCM audio signal with a low latency of 0.8 milliseconds. Anti-radar interference function The 1.2-GHz band is also used by radars. We have installed an anti-radar interference function in the wireless microphone system and achieved stable audio transmissions with it. Implementation as ARIB standard The transmission method of the wireless microphone using OFDM has been adopted as a standard by the Association of Radio Industries and Businesses (ARIB) and can now be used in the production of broadcast TV programs and in theaters and halls. We will conduct experiments intended for proving operability of the microphone to encourage early adoption of the frequency shift. Professional wireless microphones that require a license are called specified radio microphones. These are widely used in various applications such as the production of broadcast TV programs and in theaters and halls. Handheld wireless microphone OFDM system: Robustness against multipath fading Receiving antennas Wireless microphone receiver Linear PCM: High sound quality, low latency Two-piece wireless microphone Anti-radar interference function Radar Interference waves Audio signal affected by interference Original audio signal of new digital wireless microphone

33 Next Generation Broadcasting Devices 28 Creating CMOS Image Sensor Overlaid with Photoelectric Conversion Layer highly sensitive 8K Super Hi-Vision Cameras We are looking into potential new image sensors that may improve the sensitivity of 8K Super Hi- Vision cameras. On display is a CMOS image sensor with photoelectric conversion layer which can operate at low voltage. High quantum efficiency 1 within visual light range As the structure enables an aperture ratio 2 of 100% and the crystalline selenium that makes up the thin photoelectric conversion layer is capable of absorbing all the incoming light, we expect to achieve a quantum efficiency of over 90%. Controlling crystal grain size to improve video quality By creating thin films and keeping the grain size of the selenium smaller than pixel size, we have succeeded in greatly reducing video noise. We plan to develop techniques to stably multiply light-generated signal charges and reduce dark current 3 to further improve sensitivity of the sensor. 1 Quantum Efficiency: rate of signal charges against one photon of incident light 2 Aperture Ratio: rate of light reaching the photoelectric conversion layers against the total amount of incident light 3 Dark Current: small electric current output from photoelectric conversion layer even when there is no incident light All incident light reaches photoelectric conversion layer Metal wiring decreases amount of light that reaches the photodiode Signal Charge Metal Wiring Light Photoelectric conversion layer (Crystalline selenium) Metal Wiring Pixel Electrode Light Insulator Signal Charge Pixel Scan Stacked photoelectric conversion layer Output Photodiode (Silicon) Pixel Scan Conventional type layer Output Structural diagram of a CMOS image sensor

34 Broadcasting Service 29 HD Technologies in The Broadcasting Station (1) video stabilizer for high-resolution camera video We have developed a video stabilizer for 4K camera video that produces shake-corrected HD video with no quality degradation. Using this equipment, it is possible to provide shake-free HD video without special shooting skills or bulky anti-shake equipment. Stabilized HD video output from high-resolution video 4K video has higher resolution than HD, and we can stabilize it to HD resolution. Using image processing for real-time shake correction Image processing is used to detect the shaking direction of the raw video, and the shake is compensated by tracking a cropping region in the shake direction. The region cropped from the 4K video (HD resolution or greater) and the shake compensation strength can be set to any value, and the processing can be performed in real time. A video stabilizer that is easy to use Since no physical anti-shake mechanism is added to the camera, this technique does not restrict the camera operator's movements or actions. It can also perform shake correction processing on pre-recorded video and be easily used in a wide variety of programs. We will make functional improvements during the field trials, and we will also investigate the application of this technique to 8K video. This technology is being developed jointly with FOR-A Co., Ltd. Shake compensation strength: The strength of the compensation that varies according to the camera shake conditions This exhibit is presented by NHK's Broadcast Engineering Department and Engineering Administration Department. 4K video Trim Correcting shake in real time HD video 4K camera HD video stabilizer Stabilized HD video output Using the image stabilizer to correct camera shake

35 Broadcasting Service 29 Big Technologies in The Broadcasting Station (2) data visualization system We are exhibiting a big data visualization system that uses the "disaster big data" resources shown in an NHK Special program. By exploiting the advantages of real-time computer graphics (CG), we have made it possible to offer a more diverse range of TV productions, and we have also made substantial improvements in terms of mobility by making it possible to check data at the receiving end, allowing the system to run even on mobile PCs. Instant visualization of big data on the order of tens of millions of items The system can process huge quantities of big data to facilitate real-time visualization. By making use of real-time CG, we are able to selectively display the required information and expand the information relating to any particular location. Can output and display 8K CG In the NHK Special program "Disaster big data file 2" broadcast in September 2013, a big data CG was displayed on an 8K monitor that was filmed along with studio footage. This high-quality display still looked good when zooming in on part of the screen, and it allowed the analysis results of the convoluted big data to be explained in an easy-to-understand manner. Broader scope for use of real-time CG By making it possible to visualize big data even on mobile PCs, we have provided reporters and directors with the ability to analyze big data by themselves. We have also expanded the range of applications where this technology can be used as a new material-gathering and production support tool, including support for presentations based on collected materials. We will continue working toward the construction of new disaster prevention tools that use big data, including the integration of Skymap functions and cooperation with Web services. Skymap: A system developed by NHK for the real-time display of information such as building names and addresses in aerial video footage obtained with a helicopter. This exhibit is presented by the Broadcast Engineering Department. Visualizing the path of vehicles based on GPS information (NHK Special, disaster big data file.3)

36 NHK Engineering System 30 Serving Utilization and Development of NHK s Technology Society with NHK s Technologies NHK Engineering System, Inc. promotes the patents and technical expertise that NHK has acquired through its R&D activities. It also seeks ways to pass on the benefits of broadcasting technologies to the general public. The following are a few of NHK's patented and ongoing researches which are open to wider application. Technologies related to 8K Super Hi-Vision Small and lightweight 8K Super Hi-Vision Single Chip CMOS camera: can be utilized for both industrial and broadcast purposes High-resolution technology: converts Hi-Vision signals into 8K Super Hi-Vision signals 8K SHV Premium Seat: an accessible alternative that requires fewer loudspeakers to play 8K Super Hi-Vision audio Technologies with potential for wider application Hybrid sensors: for a new approach to virtual studio production Making program sound more intelligible: to assist the elderly Coupling Loop Interference Cancellers for VHF-Low Multimedia Broadcast: enables SFN on-air relays Technology transfer of patents and know-how A consultation counter on NHK's patents and expertise is open in the exhibition area. For details and consultations on NHK's patents and other technological know-how, contact : NHK Engineering System, Inc Kinuta, Setagaya-ku, Tokyo TEL (03) FAX (03) URL: SFN (Single Frequency Network): where several broadcasters use the same frequency to cover a wide area. 8K Super Hi-Vision Single-Chip Camera Virtual Studio using Hybrid sensors 8K SHV Audio Premium Seat

37 8K Super Hi-Vision 31 Specialized 8K Super Hi-Vision "Theater Camera" for shooting stage performances A wide variety of productions are ongoing to boost the line-up of 8K Super Hi-Vision (8K SHV) contents. STRL has developed a new 8K SHV "Theater Camera", which has higher sensitivity to light and produces less audio noise, and we used it for the live public screening of Teatro alla Scalla's opera performance "Rigoletto" in Tokyo last year. Luminous sensitivity improved by using larger image sensor and signal processing Theater performances tend to be underlit and the luminous sensitivity of conventional 8K SHV cameras was insufficient, often resulting in video noise. The new Theater Camera incorporates a large 2 1/2-inch full-resolution image sensor and signals are processed with a pixel addition method which reduces video noise. Camera sound noise reduced by better heat dissipation Conventional 8K SHV cameras are too noisy to be used in theaters during live performances. By improving the camera s heat dissipation mechanism, sound noise ha s been minimized and is now as quiet as other professional equipment used in music halls. It is now possible to shoot plays, operas, and ballets in quiet halls. Live public screening of Teatro alla Scalla s opera performance Teatro alla Scalla s Rigoletto performance at NHK Hall last September was shot using the 8K Theater Camera and transmitted live to two public screening venues in Tokyo and Sendai. The immersive experience made distant viewers feel as if they were watching the performance at NHK Hall. The quiet and sensitive new camera will be used to add theater performances besides opera to our line-up of Super Hi-Vision programming. We will continue to improve its luminous sensitivity and operability. Lens 33 megapixel CMOS image sensor Pixel Addition R Signal process for R-ch. Pixel Addition G1 Pixel Addition G2 Signal process for G-ch. Pixel Addition B Wavelength-division Multiplexing CCU Signal process for B-ch. Camera Head Theater Camera Block Diagram of the Theater Camera

38 Fusion of Broadcast and Broadband S1 Experience Hybridcast our latest services Here you can experience NHK s Hybridcast service that was launched last year. This exhibit showcases the services that NHK is currently broadcasting and presents examples of field trials conducted by commercial broadcasters. Broadcast services are enhanced with Internet technology NHK Hybridcast is a new television service that links broadcasting and the Internet. Starting with news and weather, you can access a variety of services such as program-related information, video clips, and the like. These services are also available on smartphones or tablets. Further enhancements Since launching Hybridcast in September last year, NHK has been expanding its functions to include other features such as on-demand video clips and services for mobile terminals. Since January, commercial broadcasters have also been conducting their own verification trials. Hybridcast-ready TV sets are becoming more widespread, and we expect that services will become more feature-rich and widely available in the future. As Hybridcast-ready receivers become more popular, we will strive to develop and enhance these services. Broadcast Internet Broadcasting station Programs Service provider Program Related Contents On Demand Content Multi-View Content Recommendation Video Highlights Program Guide Start Over Service Interactive Programming Concept and service examples of Hybridcast

39 Fusion of Broadcast and Broadband S2 The Integrated Service by Broadcasting Stations and Internet Service Companies world of broadcasting extends to the Internet Information about TV programs and commercials is being spread via a diverse range of Internet services, including social network services, websites, and smartphone apps. This exhibit introduces examples of services that provide viewers with a richer and more convenient experience through collaborations between broadcasting stations and Internet service providers. Program-related information and user-generated information This is information related to TV programs and commercials provided by broadcasting stations and advertising agencies. It is transmitted via the Internet between companies together with information from messages posted by users of SNS websites; a wide variety of services that take television broadcasts as a starting point can be offered. We will develop service models for collaborations between broadcasting and Internet services in order to create new services and businesses by allowing information held by broadcasting stations to be effectively used by service providers over a wide range of applications. SNS (social network service): An Internet service that has communication functions that promote and support connections between people. Information about TV programs and commercials Advertising agency Broadcasting station Advertiser E-commerce business Viewers & Internet users TV information sites and smartphone app providers Provider of information related to TV programs or commercials (metadata) SNS operator Companies make use of this information in their programs/services Store User-generated information Measure the growth of social network activity associated with TV broadcasts Measurement/ survey company Collaborations between broadcasting stations and diverse service providers

40 Museum of Broadcasting M The NHK Broadcast Museum Olympic Games of 50 Years Ago The 18th Olympic Games were held in Tokyo in This was the first time the games had been held in Asia, and although only 11 years had passed since TV broadcasting had started in Japan, everyone involved with broadcasting, starting with NHK, pulled together to broadcast the Olympics on TV. As a result, it became a national event that united Japan through the media. In particular, the final of the women s volleyball competition faced off the Japanese team, known as The Oriental Witches, in a fierce battle for the gold medal with the Soviet team and had an audience rating of over 85%. The game remains a part of media history even now. The Tokyo Olympics demonstrated to the world the height of Japanese broadcast technology and was also a golden opportunity for the Japanese TV industry to spring out into the world. The Institute of Technology (now the NHK Science and Technical Research Laboratories) carried out research and development and contributed to the Olympic broadcasts. Development of color cameras ensured success of live relays We developed a 2IO separate luminance color camera, designed to have increased resolution, reduced noise, and also a compact form. It vividly conveyed the atmosphere of the Opening Ceremony. The world s first overseas live broadcast We made live overseas broadcasts to the USA for the first time ever using the geostationary satellite Syncom3. Development of various production equipment We developed equipment, such as the world s first slow motion replay, live-broadcast equipment that was used over the entire marathon course, a 1/2 compact vidicon camera, and a close-talking microphone, to convey the atmosphere of the Olympics outside of Japan. Display from the NHK Museum of Broadcasting. Developed broadcasting equipment 1/2 compact vidicon camera(1963) 2IO separate luminance color camera (1964) Sanken ML-1 close-talking microphone (1964)

41 Service for Viewers J Digital Broadcasting Reception Consultation Desk Advice on digital broadcasting receiver systems and home networks We welcome your questions on digital broadcasting reception methods and home networks. Digital broadcasting reception and receiver systems We can answer your questions about the various digital broadcasting reception methods and receiver systems available for individual households and housing complexes. We can also explain Cable TV reception and NHK's hybrid services that are offered through broadcasting and communications channels. Home networks and digital broadcast reception We can explain the types of Internet equipment that can be installed in the home and tell you how to set up your own home networks. We will continue to work at improving our services for you, the audience. This exhibit is presented by the Engineering Administration Department. Terrestrial broadcasting Satellite broadcasting Broadcasting station Network audio Smart digital TV Personal computer communi Collaborative services Built-in HDD recorder Blu-ray Internet Cloud computing Router Home Network (In-Home LAN) Network printer Smartphones & tablet devices NAS server Handheld game console Video camera Smart home appliances

42 Poster Exhibit P1 Interactive Object Extraction Method for "Video Bank" System To utilize video footage more effectively and efficiently We developed a new video footage management system, called "Video Bank", which automatically adds useful metadata, for video retrieval and compositing purposes, as the footage is being stored in the system. For compositing, the system interactively extracts the desired video object area from the footage. P2 Improved Method for Measuring MTF of 4K/8K Cameras It is important to measure the spatial resolution characteristics of 4K/8K cameras in order to fully exploit their capabilities, which should be well in excess of current high-definition (2K) broadcast cameras. The conventional evaluation methods measure the characteristics at a specific spatial frequency, either in the vertical or horizontal direction of the image. We have developed a new method that measures camera resolution characteristics at all frequencies in all directions in a single measurement. P3 Effects of 3-dimensional Reverberation on Spatial Impressions in 8K Super Hi-Vision Sound Production Simplified process of sound production to create enhance sense of presence and reality In order to create sound for 8K Super High-Vision in accordance with the sound engineer s intentions, we are studying a method where the sound of each 22.2 channel is not controlled independently. Here, we illustrate the relationship between spatial impression and the reverberation parameters in 22.2 multichannel sound. P4 Non-native Japanese Speakers Understanding on News Specific Expressions Finding the best expressions for easy-to-understand news reports News reporting in Japan features many expressions that are not used in everyday conversations. We have studied how well these expressions are understood by international residents in an effort to deliver easy-to-understand news in Japanese to our foreign audience. P5 Estimating Subjective Mental States from Brain Activity Obtained During TV Viewing Exploring a new program content evaluation technology We are studying a method that estimates the mental states of viewers while they are watching a program by analyzing their brain activities. By using machine learning, this method can estimate the viewer's "readiness to laugh" by automatically extracting a characteristic pattern from the measured brain activity.

43 P6 Directionality Control Method of Light Beam through Sub-micron Dielectric Structures Towards the 3D displays of the future We are studying optical devices for future 3D displays. We have developed a device consisting submicron-sized dielectric cylinders that has capability of deflecting a light beam in controlled direction. P7 3D Integration Technology for Imaging Device For devices that have both ultra-high resolution and high-frame rates Looking beyond 8K super-high definition television, we are studying a imaging device with a 3D structure that can meet the ultra-high resolution and high-frame rate requirements of future systems. The device has a digital processing circuit formed beneath the photodetector and reads out signals from all pixels in parallel. P8 Continuously stacked Technology in Organic Image Sensors To create compact, high resolution single chip color cameras We are studying organic films that convert each of the 3 primary colors of light into electronic signals and an organic image sensor made by stacking organic films and transparent readout circuits. The continuously stacked technology aims to improve the resolution image sensors by eliminating defocusing for each color. P9 Viewing Behavior on Time Shift Zapping System of Years of Broadcasting Contents Analyses aimed at providing new VOD services in the age of cloud computing Time-shift viewing allows you to watch past programs anytime you want. We are studying ways to provide this broadcasting service of the future. Viewing experiments of the service where viewers can zap in the time direction through several years' worth of past television programs were conducted, and the findings on the viewing behaviors are reported here. Interactive Exhibit T1 Haptic TV Touch me if you can Clutch the little bag while you watch the T.V. Can you feel the popcorn popping in the pan? T2 Augmented TV Look at the T.V. through the tablet and see the cartoon characters pop out!

44 Research Presentation To be held on May 29 (Thu.) Research presentation 1 Enhancement of Integrated Broadcast-Broadband System Toward The Advanced Hybridcast Services Hisayuki Ohmata (Integrated Broadcast-Broadband Systems Research Division) 10:20 a.m. Hybridcast is a technology platform for integrated broadcast-broadband services that provide viewers with a new kind of TV experience. In September 2013, NHK launched a new TV service, "NHK Hybridcast". STRL has conducted research and development for enhancing Hybridcast technologies to realize more advanced and diverse services. We are also contributing to standardization activities so as to extend technical specifications of Hybridcast. The presentation begins with the technical concept of a non-broadcast-oriented managed application which allows third party companies other than broadcasters to provide Hybridcast services. This type of application can provide various cross-channel services. We introduce examples of these services and a system model for secure and smooth delivery of application service by broadcasters and third parties. Other elemental technologies have also been developed for advanced Hybridcast services. We describe enhancements for frame-by-frame synchronization between broadcast and broadband, video streaming and recorded broadcast program. In conclusion, we present our efforts to launch advanced Hybridcast services and our vision for the 8K Super Hi-Vision era. Research presentation 2 Development and standardization of an optical interface for 8K Super Hi-Vision Takuji Soeno (Advanced Television Systems Research Division) 10:40 a.m. NHK has been researching 8K Super Hi-Vision (8K SHV) as a next-generation broadcasting system. We are developing full-specification 8K SHV equipment with 8K resolution ( pixels), a 120-Hz frame frequency, 12-bit coding format, and a wide-color gamut. In many cases, it has been necessary to use numerous coaxial cables to transmit 8K SHV signals, which has presented issues in practical aspects. There is consequently a need for the development and early standardization of an interface that can transmit 8K SHV signals using a single cable. In this presentation, we introduce a new 8K SHV optical interface that was standardized as ARIB STD-B58 in March First, we show the requirements of the 8K SHV interface for connecting 8K SHV video devices. Then, we explain the new mapping method, based on multi-link 10-Gbit/s streams, and discuss the 8K SHV signal transmission experiment. Finally, we present the development of a full-specification 8K SHV camera system, recording equipment and display incorporating the optical interface. 11:00 a.m. Research presentation 3 Using a multi-viewpoint camera to generate integral 3D images Kensuke Ikeya (Three-Dimensional Image Research Division) At STRL, with the aim of developing a next-generation ultra-realistic broadcasting system, we are researching integral 3D television, which produces natural-looking images that can be viewed without special glasses. Integral 3D television generates spatially three-dimensional images by using a combination of a lens array (consisting of small lenses) and a high-resolution display to recreate the rays of light emitted from a visual object. In integral 3D television, a special dedicated camera is used to produce integral 3D images. When capturing visual objects that are distant from the camera, like players on a sports field, it has been difficult for the special dedicated camera to generate 3D images. In this presentation, we introduce a method that allows integral 3D images to be formed of remote visual objects by using a multi-viewpoint camera consisting of multiple ordinary TV cameras to capture video from many different directions (multi-viewpoint video). In this method, multiple 3D models are produced by using a probability-based depth distance estimation method to suppress the estimation errors. After converting these models into images for integral 3D video generation, they are synthesized together to reduce the regions where the camera view is occluded by other objects, thereby producing a defect-free integral 3D image. The usefulness of this method is demonstrated with an example of integral 3D content produced from a broadcast sumo contest.

45 Lecture To be held on May 29 (Thu.) Lecture 1 Expectations for 8K Super Hi-Vision to be introduced in the medical field Ryozo Nagai, President, Jichi Medical University 11:30 a.m. Development of 8K Super Hi-Vision is under way in earnest. Clearer pictures can lead to new discoveries and change people s awareness and behavioral patterns. Humans have developed telescopes and microscopes so that they can see things more vividly. But those instruments only serve the needs of individual observers. For many people to share the same experiences, high-resolution images are indispensable. High-resolution images have a broad range of applications in medical practice. They can be immediately applied to telemedicine and medical education. As Japan suffers from shortage of clinical pathologists, a limited number of experts are available for diagnosing specific cancers across the country. A combination of information network and high-resolution images would be of great help to medical services not only in remote areas but also in urban communities. High-resolution images are highly useful in learning surgical techniques. In recent years, an increasing number of operations entail repairing tiny blood vessels or nerve fibers. Using magnifiers or microscopes for that purpose has become common practice in operating rooms. These devices allow surgeons to clearly view the surgical areas, but assistant members in the room have to depend on TV monitors to understand how the operation is going and learn the surgeons techniques. Improving the images resolution will have huge educational effects on those novice surgeons, allowing them to watch very fine surgical threads and needle tops that can t be spotted on current high-definition TV monitors. Surgeons themselves will discover many things they don t notice during the operation by watching their performances on video screens afterward. In addition, pictures of magnified surgical areas will help identify the lesion locations. Endoscopic examination has also advanced considerably. Endoscopes currently used to examine the stomach or intestines allow physicians to discern capillaries thanks to their enhanced resolution. If observation at a cellular level is made possible, doctors can detect cancers at an earlier stage. But many technical issues need to be cleared before endoscopes can provide sharper images to be displayed on an 8K Super Hi-Vision monitor. Japan is on the leading edge of developing endoscopic technology. In order to raise standards and encourage innovation in medical services, 8K Super Hi-Vision should de designed to work in conjunction with endoscopes as soon as possible. Lecture 2 Seeing both the forest and the trees: Techniques for analyzing big data 0:10 p.m. Tomoyuki Higuchi, Executive Director of Research Organization of Information and Systems Director-General of the Institute of Statistical Mathematics The advent of big data has ushered in a new era. The whole range of our life every activity, movement and event that is taking place can be digitized and recreated in the cloud now. Big data can create value for businesses in two main ways; it makes every business function and service smarter, and it helps discover innovations and thereby promotes effective investment toward the future. Similarly, big data can benefit the public sector by improving each service recipient s satisfaction on the one hand, and by helping discover rare, yet important, social factors on the other. Many technical challenges we face in using big data come from the demand that we achieve both of these contrasting goals. For example, no matter how huge the amounts of data might be, sampling would allow us to capture the general picture with some degree of accuracy. Yet, this common statistical procedure would most likely lead us to miss rare phenomena. This means that in order to make use of big data, we need techniques that let us see both the forest and the trees. Big data techniques and technologies can be divided roughly into three elements. They are visualization, techniques for analyzing big data, and big data technologies. Visualization can be categorized as a technique of exploratory data analysis, but with the advent of big data, it has come to play a crucial role in Nowcasting (capturing the present). Big data technologies hold the key to resolving problems resulting from increases in data amounts and communications traffic. Advanced techniques for data analysis are essential when we deal with data incompleteness, which is a state where data cannot give complete information we aim to get. There are many, diverse examples of data incompleteness. Among them are such common problems as missing data and the presence of outliers. Some others, including unstable problem of inverse analysis, are inevitable results of the way inferences are made. In this lecture, I would like to describe issues that frequently emerge in big data analysis, and introduce current trends in studies on data analysis, including imputation, linkage, and stream computing.

46 8K Super Hi-Vision Theatre May 29 (Thu) June 1 (Sun) 10:00 am 5:00 pm at NHK STRL Auditorium (Restricted to one viewing per entry) Final screening on each day is scheduled to end at 5:00 pm. There will be no screening between 10:00 am and 1:30 pm on Thursday, May 29 due to lectures and research presentations Program: Highlights from Teatro alla Scala s Rigoletto in Tokyo, 2013 Last year, Italy s Teatro alla Scala celebrated the bicentennial of Giuseppe Verdi s birth with a performance of his masterpiece opera Rigoletto in Tokyo. Our 8K screening will deliver the best moments from the act in a 10-minute highlight. Especially designed for high performance in underlit and calm environments, NHK s original 8K Super Hi-Vision Theater Camera has captured one of the world s most renowned opera in detail, from the extravagant costumes to each artist s subtle signs of emotion. Come and experience our immersive 33 megapixel high-resolution video accompanied by a 22.2 multichannel 3D sound system, and it will instantly take you on a virtual trip to the greatest theater in the world. photo : Marco Brescia / Teatro alla Scala Event dates May 31 (Sat) and June 1 (Sun) 10:20 am - 3:20 pm Guided tours STRL researchers will guide you through the exhibits. Scheduled 16 times a day. Duration: approx. 1hr 10:00 am - 4:30 pm Stamp - athon Look for rubber-stamps of various characters while you enjoy the exhibits. Stamp them on the right answers! 10:30 am - 4:00 pm Kick-the-8K-Target-Out Come to our Lawn Square to take an aim at the 8K mark and give the soccer ball a full kick! Please note each event has limited capacity.

47