Overview of the Hybridcast System STRL is conducting research on Hybridcast, a technology platform that uses communications networks to enhance broadcast services. Hybridcast makes it possible to create various services with many functions. Examples include a service that enables viewers watching video on demand (VOD) programs related to broadcast programs, a service that allows viewers to customize their viewing experience, a service that links with social networking services (SNS), and a service that links smartphones and tablets to TV. This article is an overview of the Hybridcast system. 1. Introduction The technological environment surrounding broadcasting and communications has been rapidly evolving. The last decade has seen the digitization of broadcasting, the spread of broadband communications, the development of Internet technology and culture, and the advent of a variety of information terminals (Figure 1). The communications world is especially undergoing rapid changes; in FY2011 in Japan, approximately 35 million households had broadband, and over 21 million new smartphones were sold. There is currently fierce international competition to develop and deploy TV that exploit the features of broadcasting and communications. The competitors in this arena include GoogleTV and Hulu in the U.S., and Hybrid Broadcast Broadband TV (HbbTV) and YouView *1 in Europe. NHK s FY2012-2014 administration plan *1 A VOD standard in the U.K. *2 Hybridcast is a registered trademark of NHK-ES. calls for the development and provision of new services integrating broadcasting and communications, and STRL is conducting research on Hybridcast *2, a platform for services that use broadcasting and communications 1)-3). Our goal is to deploy Hybridcast within two years, and we are now standardizing and building the systems that will be used for testing and demonstrating Hybridcast. This article describes the background and objectives of Hybridcast. It presents examples of Hybridcast services, system prototypes, and future plans. 2. Hybridcast Development More than ten years have passed since digital broadcast satellite (BS) broadcasts began. The digitalization of broadcasting made various services possible. One of these services is the electronic program guide (EPG). The EPG displays a program guide on the TV screen that immediately reflects program scheduling changes such as emergency bulletins or overtime in sports coverage. Moreover, as viewing styles diversified, the EPG eventually became an essential means of recording programs. The reason is that digital broadcasting specifications are created through the cooperation of broadcasters and receiver manufacturers, and broadcasters provide the service information (SI) for the EPG in a common format. Broadcasters have a duty to provide viewers with highquality programs and reliable information, and they must make their services more convenient. To fulfill our duties in a world where broadcasting and communications are becoming more and more integrated, we decided to cooperate with other broadcasters and receiver manufacturers to consolidate a system that conveys the Digitalization of Broadcasting - High quality, Reliable, content delivery over a wide area - High capacity, low cost - High rate of adoption Internet Technology and Cultural Development - Cloud computing - Social networking - User-generated content Spread of Broadband - Personalization - On-demand program viewing - 35 million contracts nationally Diversification of Information Terminals - Mobile phones, smartphones - PCs, tablets Attractive services adapted to the times, maximizing utilization of these elements Figure 1: Environment of broadcasting and communications 2
Feature broadcasters intentions correctly and reliably to viewers and enhances the capabilities of digital broadcasting by exploiting the Internet. As such, we set a goal of implementing Hybridcast in the short term. 3. Hybridcast Technical Specifications Figure 2 illustrates concept of Hybridcast, our technology platform for creating advanced broadcast services using networks. In particular, functions (1) to (4) in the figure are used to provide new services that integrate broadcasting and communications. (Note that this issues article, Technologies supporting Hybridcast, presents the technical specifications for Hybridcast 4).) The following objectives were incorporated into the specifications of Hybridcast. (1) Change existing broadcast formats as little as possible We should minimize the changes that have to be made to existing broadcast formats by taking into consideration compatibility with current digital receivers and the effects on broadcast facilities. (2) Use formats that are compatible with international standards and communications Broadcasting cannot reach everywhere, but communications networks have no borders. We studied formats from a global perspective. HTML5 5), an enhancement of HTML, which is the language used to describe Internet applications, is currently undergoing standardization at the World Wide Web Consortium (W3C). We chose to use it as the environment for running applications in the draft Hybridcast specifications. (3) Enable users and service providers to provide a variety of services (applications) Information distributed by service providers other than the broadcasters could contain content that is not appropriate for broadcasting. Considering the public nature of broadcasts and childcare issues, a mechanism to prevent inappropriate content from being displayed should be provided to ensure that the quality of content can be managed. (4) Enable broadcasting and communications content to be synchronized for composition and presentation Services such as multilingual captioning, speedadjusted voice audio for the aged, and multi-viewpoint video can be implemented by synchronizing information from communications networks with broadcast programs. (5) Enable new services to be developed as the needs of viewers change and technology advances Cloud computing is an effective way to foster services that can respond to changing viewer needs and make use of newly developed technologies. For example, by performing media format conversions, summarizing or translating content, and other types of cloud computing, new services can be provided without the viewer having to purchase a new receiver. This requirement reduces the cost of receivers and helps to integrate new communications services such as social networking services (SNS) *3 with broadcasting. (6) Integrate and harmonize the cultural differences *3 A communications service on the Internet. Provide information related to broadcasts Network Additional content for broadcast programs Social networks On-demand programs Program recommendations (1) Functions for linking social and personal information to broadcasts, only possible using communications (3) Functions integrating communications content with broadcast programs, making them more interesting and understandable Terminal-linking services View program information and control the TV from a second screen (4) Functions for providing reliable information (2) Functions linking TV to mobile terminals and PCs, making programs more convenient and enjoyable Figure 2: Hybridcast conceptual diagram 3
between broadcasting and communications TV is primarily a passive medium for viewing that can be enjoyed together with the whole family. On the other hand, communications media such as the Internet are basically oriented to individuals, allowing them to access the information and content whenever they like. Integration of broadcasting and communications has been talked about for a while, but no significant success has appeared yet. It will be very important to harmonize the differences in between broadcasting and communications cultures as much as possible in order to create services that link them. Let us discuss some of the differences between broadcasting and communications cultures. As mentioned earlier, TV is basically a passive medium. By watching TV, viewers can learn about the major events occurring around the world. They can get excited watching live sports coverage, or have a rousing discussion about what happened on yesterday s dramas. TV are simple to operate and do not need the upgrading or other maintenance required by PCs. They are easy, inexpensive appliances that can be used for ten years or more after purchase. The quality of content is guaranteed by the broadcasters, so broadcasting is a very safe and secure medium. On the other hand, communication using PCs and smartphones is an active medium. Hardware is replaced and software is upgraded regularly and frequently. There is an enormous amount of information that can be obtained from the Internet, but the content and quality is extremely varied and wide ranging. The fact that information on the Internet is basically not checked by anyone has its attractions, but users must decide for themselves whether the information is reliable. Hybridcast is an attempt to harmonize reliable broadcasting with the detailed and interesting information available on the Internet. Hybridcast s technical specifications include a mechanism to control the degree of this harmonization in order to handle needs and new situations that may arise, changes in viewing style, and the various business models of each broadcaster. As an example, displaying an application window over the image of a broadcast program could potentially violate the copyright of the broadcast program, so we have included the ability to specify whether overlays are permitted or not for each program. We have also strengthened functions for linking TV with other terminals such as mobile devices and PCs. The ways of watching and using TV are diversifying and vary with the size of the screen. For example, the whole family can watch a program on the large-screen TV, while information tailored to each family member can be displayed on the screens of tablet computers. 4. System Structure of Hybridcast Hybridcast includes the following system elements (Figure 3) 6)7). (1) Broadcast station Application launch information *4 and presentation policy *5 information are broadcasted together with the digital broadcast signal. Program-related information and meta-data enabling broadcasters to tailor their services to the contracts with the viewer *4 Various information is needed to launch an application, including information about where to obtain it, and how to it and stop it. *5 Special information is needed to control the position of the content on a screen. For example, it is possible to send control information to make the screen show an L-shaped banner for scrolling news and emergency bulletins around a broadcast program. Broadcast station Program-linking metadata, etc. Broadcasting (high quality, high reliability, multi-destination) Presentation policy Application launch information Internet Broadcast program Net services linked and cooperating with broadcast programs Hybridcast receiver Makes possible a variety of services based on broadcast programming HTML5 execution environment Service provider Application content Communications (handling requests on a per viewer basis) Use linked with mobile terminals and PCs Figure 3: System structure of Hybridcast 4
Feature and the service provider are also provided. (2) Service provider Service providers create and distribute applications. The definition of service providers includes broadcasters and other businesses. (3) Hybridcast receiver The main functionalities of the receiver are: a functionality to run HTML5 applications distributed by service operators and to process content that uses communications, such as VOD. a display functionality that synchronizes the presentation of broadcast and communications content. a functionality for linking with other terminals. a functionality to enable applications to control the display. 5. Service Examples and Prototypes We presented the Hybridcast concept at the 2010 NHK STRL Open House; the exhibit included a receiver implemented on a PC 2). The following year, we exhibited prototype receivers from different manufacturers and showed some of the things made possible by Hybridcast (Figure 4) 8). At the 2012 Open House, we exhibited a prototype receiver including an HTML5 browser and showed how various services can be implemented with it (Figure 5). There are numerous services that use functions (1) to (4) in the Hybridcast overview diagram (Figure 2). (1) Services that use social and personal information gained through communications. (i) Social TV service This service links viewers of broadcast programs through social networks on the Internet. For example, viewers who have joined an SNS can post their opinions and impressions of a program they are watching and participate in conversations on the SNS 9). Friends watching the same program can exchange ideas by using their tablet computers. (ii) Program recommendation service This service recommends and provides programs related to the program currently being viewed. The programs are stored in a large VOD library, and the service makes suggestions according to their popularity on the SNS and the viewer s preferences and needs. Recommended programs can be bookmarked for later viewing. (iii) Personalized program-spot *6 presentation service This service recommends programs according to the viewer s preferences instead of showing traditional program spots for the current broadcast. For example, a sports program spot can be delivered through communications channels and presented automatically to a person who watches a lot of sports (Figure 6). This service infers the viewer s preferences from their TV viewing history and other personal viewing traits. It is being developed in cooperation with NTT. (2) Services that allow programs to link the TV to mobile terminals and PCs. Figure 4: Prototype Hybridcast receivers from different manufacturers *6 A program in which a broadcaster introduces its programs. Multilingual closed caption service Program recommendation service Menu screen Menu screen Playback on TV Social TV service Figure 5: Examples of Hybridcast service Terminal linking service 5
Sports program spot Search suggestions displayed automatically Mr. A: Interested in sports Figure 6: Personalized program-spot presentation service (i) Terminal linking service This service displays information on a mobile terminal instead of on a large TV screen. In the service example shown in Figure 7, viewers can search Web content by using keywords and that content is automatically presented on the screen as the TV program progresses. Mobile terminal features can be exploited in various ways. For example, a tablet computer could be used to operate a TV or enter text in an application. (ii) User authentication service This service enables users of social services to login from their mobile terminals. This eliminates the need for login operations on the TV. Mobile terminals Program title, overview and keywords matched to the progress of the show are displayed while viewing Link Web search results for selected keywords aredisplayed Figure 7: Example of a service provided to mobile terminals Social and other services requiring user authentication Enter ID and password Mobile terminal Authentication linking TV Figure 8: Example of a service requiring user authentication 6 Broadcast Technology No.51, Winter 2013 C NHK STRL
Feature Figure 9: Example of a multi-viewpoint service using images cropped from SHV have an excellent user interface and have relatively easy login operations. Thus, by having the mobile terminal pass login information to the TV, it can be used to access various personalized services without needing to perform any specialized operations (Figure 8). (3) Services that integrate communications content with broadcast programs. (i) Customization service This service provides content related to the program being broadcast through communications channels. It takes the needs of the individual viewer into account and synchronizes the presentation with the broadcast program. Examples include multilingual caption services for foreigners or speed-adjusted audio for the elderly 10). At the 2010 NHK STRL Open House, we showed that captions provided by communications networks could be displayed with almost no noticeable delay. (ii) Multi-viewpoint service This service distributes video segments related to the broadcast program over a communications network and composes them with the program for display. It uses a high-precision synchronizing technology Figure 10: Example of a multi-viewpoint service showing images taken by multiple cameras that can synchronize content with single-frame precision 11). Figure 9 shows an example of the multi-viewpoint service. The multi-view segments are cropped and zoomed parts of a Super Hi-Vision image (SHV) and are overlaid on the main image. It is also possible to create multi-viewpoint video made Figure 11: Example of a CG synthesis service (sign-language CGs) Example of application display control Receives emergency earthquake bulletin Display of the application is suspended Figure 12: Example of a service to control the presentation of an application (giving priority to displaying emergency bulletins) 7
from images taken by multiple cameras (Figure 10). Viewers can select the camera video they want to see and watch it continuously overlaid on the program video. A 3D-HDTV video service could also make use of high-precision synchronization technology. For example, HDTV video for the left eye could be sent in the broadcast signal while the video for the right eye is distributed over the communications network. The HDTV video could be viewed on existing TV through the broadcast, while receivers supporting Hybridcast and 3D displays could be used to view the 3D-HDTV video. This service is being researched in cooperation with NTT. (iii) Computer graphics synthesis service This service displays broadcast programs synchronized with computer graphics (CG). For example, sign-language CG video generated using TV program Making Language (TVML) 12) can be sent over a communications network to hearing-impaired viewers (Figure 11). This service could also be used to display graphics such as off-side lines in soccer. (4) Services using functions that provide information accurately and reliably (i) Giving priority to emergency bulletins 13) When a terminal receives a bulletin such as an emergency earthquake report while an application is using its display, the application must stop using the display and the emergency information must be displayed reliably (Figure 12). To do this, the environment must always be able to control the display and certify and permit applications to run. 6. Conclusion Besides the venue of the NHK STRL Open House, we have exhibited Hybridcast at many trade shows and conferences within and outside Japan, including NAB2011, CEATEC, and IBC2011 14). In December 2011, we started a working group at the Japanese standardization organization, the IPTV Forum, to decide on technical specifications for services linking broadcasting and communications using HTML5. This working group has many participants from manufacturers, broadcasters, and communications providers. The HTML5 specifications being studied there are closely related to the Hybridcast specifications. In the future, we will create system prototypes and conduct tests based on the specifications determined in cooperation with these related enterprises and organizations, and we will continue to work toward early deployment of this technology. (Hiroyuki Hamada) References 1) Katoh: Toward the Realization of Hybridcast, NHK STRL R&D, No. 123, pp. 56-63 (2010) (http:// www.nhk.or.jp/strl/publica/rd/rd123/rd123-j.html) (Japanese) 2) Matsumura, Kanatsugu: Overview and Technology of HybridcastTM, NHK STRL R&D, No. 124, pp. 10-17 (2010) (http://www.nhk.or.jp/strl/publica/rd/ rd123/rd123-j.html) (Japanese) 3) Matsumura: The Development of HybridcastTM: Technical Requirements for the Enhancement of Broadcast/Communication Hybrid Services, NHK STRL R&D, No. 129, pp. 24-31 (2011) (http:// www.nhk.or.jp/strl/publica/rd/rd129/rd129-j.html) (Japanese) 4) HybridcastTM Technical Specifications Overview, http://www.nhk.or.jp/strl/hybridcast/ (Japanese) 5) HTML5 (working draft), http://www.w3.org/tr/ HTML5/ 6) Mitsuya, Matsumura, Baba, Fujisawa, Takechi, Yamamoto, Kanatsugu, Hamada: Realizing HybridcastTM: Studying Specifications and Prototyping Basic Functionality, ITE Winter Conference, 10-3 (2010) (Japanese) 7) Kanatsugu, Mitsuya, Matsumura, Baba, Fujisawa, Takechi, Yamamoto, Hamada: Proposal of Hybridcast System, ITE Journal, Vol. 35, No. 7, CE2011-15 (2011) (Japanese) 8) NHK STRL Open House 2011, Hybridcast, http://www. nhk.or.jp/strl/open2011/tenji/02_e.html 9) Ariyasu, Fujisawa, Kanatsugu: Message Analysis Algorithms and Their Application to Social TV, 9th International Interactive TV and Video Conference (EuroITV 2011), ACM, pp. 1-9, Jun. (2011) 10) Matsumura, Kanatsugu, Hamada: Prototype Program-Extension Service with Broadcast- Synchronized IP Streaming, ITE Annual Conference, 10-5 (2010) (Japanese) 11) Baba, Matsumura, Mitsuya, Takechi, Kanatsugu, Hamada: Implementing a Receiver with built-in Synchronized Display of Broadcasts and IP Content, ITE Winter Conference, 10-1 (2010) (Japanese) 12) Kaneko, Inoue, Kato, Umeda, Hiruma, Nagashima, Development of a Sign-Language Animation- Generating Editor, HI Society Journal, Vol. 13, No. 5, p. 17-20 (2011) (Japanese) 13) Otsuki, Ohmata, Fujii, Majima, Inoue: A Method of Controlling Presentation for Applications in Services Linking Broadcasting and Communications, ITE Annual Conference,11-11 (2011) (Japanese) 14) Baba, Matsumura, Mitsuya, Takechi, Kanatsugu, Hamada, Katoh: Advanced Hybrid Broadcast and Broadband System for Enhanced Broadcasting Services, NAB, Broadcast Engineering Conference, pp. 343-350 Apr. (2011) 8