Personal Mobile DTV Cellular Phone Terminal Developed for Digital Terrestrial Broadcasting With Internet Services

Personal Mobile DTV Cellular Phone Terminal Developed for Digital Terrestrial Broadcasting With Internet Services ATSUSHI KOIKE, SHUICHI MATSUMOTO, AND HIDEKI KOKUBUN Invited Paper Digital terrestrial broadcasting in Japan specifies a service profile not only for fixed receivers in the home, but also for personal mobile terminals such as acellular phones and PDAs. Services for personal mobile terminals are expected to become more attractive with sufficient functionality and flexibility by utilizing the network connectivity of mobile terminals. In order to verify the possibilities of digital television broadcasting (DTV) services for personal mobile terminals and clarify technical problems at the commercial level, we developed a prototype mobile terminal capable of receiving digital terrestrial broadcasting in addition to the Internet. This paper describes the assumed DTV service for personal mobile terminals and the technical architecture of the developed mobile terminal with regard to the hardware and the software. Several experiments using the developed mobile terminal were conducted and the performance was evaluated. Keywords Cellular phone, digital terrestrial broadcasting, integrated services, Internet streaming, personal mobile terminal. I. INTRODUCTION Terrestrial digital broadcasting was launched in the Tokyo, Osaka, and Nagoya areas of Japan in December 2003. The broadcasting is composed of three different service profiles as described in the following. The first service is for home terminals. It corresponds to the high-definition television (HDTV) service for typical fixed receiver terminals. The second service is for mobile terminals. A typical mobile terminal assumed in this service is car navigation equipment. The third is for personal mobile terminals such as PDAs or cellular phones. Manuscript received February 27, 2005; revised August 23, 2005. A. Koike and S. Matsumoto are with KDDI R&D Labs., Inc., Saitama, Japan (e-mail: koike@kddilabs.jp; matsumoto@kddilabs.jp). H. Kokubun is with the Science and Technical Research Laboratories (STRL), NHK (Japan Broadcasting Corporation), Tokyo 157-8510, Japan (e-mail: kokubun.h-ek@nhk.or.jp). Digital Object Identifier 10.1109/JPROC.2005.859699 Of the three services for digital television broadcasting (DTV), the service for personal mobile terminals is expected to become an attractive service with sufficient functionality and flexibility by utilizing the network connectivity of mobile terminals [1]. In order to verify the possibilities of the DTV services for personal mobile terminals and examine technical problems at the commercial level, we developed a prototype mobile terminal, which is a cellular phone type, capable of receiving digital terrestrial broadcasting in addition to the Internet service. This paper describes the assumed DTV services (so-called integrated services) for personal mobile terminals and the technical architecture of the developed mobile terminal with regard to the hardware and the software. Several experiments using the developed mobile terminal were conducted and the performance was evaluated. II. NETWORK CONFIGURATION Fig. 1 shows an assumed network configuration where the integrated services can be provided. The service network includes several items of server equipment. The main role of each server is summarized below. A. DTV Content Server The DTV content server distributes DTV contents in an MPEG-2 transport stream (TS) format. The server is located in a broadcasting station. The DTV content is transmitted based on the standard protocol stack shown in Fig. 2. Broadcasting Markup Language (BML) is used to describe a hypertext data, which is based on XHTML and several extensions specific for the DTV service. B. Internet Content Server The Internet content server provides streaming contents in video as well as typical web contents prepared for cellular 0018-9219/$20.00 2006 IEEE PROCEEDINGS OF THE IEEE, VOL. 94, NO. 1, JANUARY 2006 281

Fig. 1. Network configuration for integrated services. Fig. 2. Transmission format of DTV contents. phone terminals. These contents can be accessed directly via an Internet connection and are also accessible from the BML in DTV contents, which includes a hyperlink to the corresponding URL. C. Program Distribution Server The program distribution server receives DTV contents and extracts MPEG-2 TS packets. These packets are converted in a streaming format preparing for the program redistribution according to user demands. D. Network Boot Controller The network boot controller manages a user profile for each mobile terminal, and sends a trigger message to an arbitrary mobile terminal. This message is utilized in order to switch the terminal operation mode (stand by mode, DTV mode, etc.) automatically based on user requests. III. PROPOSED APPLICATION MODEL A prototype terminal was developed to verify the possibilities of integrated service for personal mobile terminals and clarify technical problems at the commercial level. Fig. 3 shows the proposed GUI layout of the developed prototype terminal. The display area is divided into two areas under the TV mode operation. The top area is allocated to display the video content of a DTV program. The bottom is allocated to display the browser output. It is commonly used by BML contents in DTV and Internet web contents. The operation mode is switched by the TV button and the Internet button on the terminal. When the TV button is pressed, the TV mode 282 PROCEEDINGS OF THE IEEE, VOL. 94, NO. 1, JANUARY 2006

Fig. 3. GUI layout of prototype terminal. is selected and DTV programs are displayed. On the other hand, pressing the Internet button initiates the Internet web access. Several examples of assumed application images are introduced in the following. A. Emergency Warning System The terminal operation mode can be switched depending on a trigger message received from a network boot controller. By using this mechanism, an emergency warning system can be implemented. This system notifies a user of special news bulletins when a disaster occurs. As a general use, this mechanism is also applicable for a TV program reservation service, where the TV play operation is automatically started in synchronization with the program situation requested by the user in advance. B. GPS Information Service By utilizing the GPS service of a cellular phone, regional information can be properly selected and provided from Internet web contents, depending on the current terminal position measured by GPS. Based on the mechanism, a weather information service for the target area is automatically determined by adapting to the terminal position. C. Assistance for Users Narration texts for the selected TV program can be obtained in many languages as Internet web contents. In the Japanese DTV service, insertion of subtitles written in Japanese is mandatory for any on-air program. However, further detailed information can be obtained with regard to the currently distributed program from the Internet web server. This application contributes to assisting users who require narration text data using appropriate languages to understand the program content. D. TV Chat Service When more than two users are watching the same TV program, the so-called TV chat services are available by mutually submitting short messages to a chat server located on the Internet. In addition to these applications, TV program navigation services such as web EPG (electric program guide) are provided. IV. DEVELOPED PROTOTYPE TERMINAL The detailed system architecture of the developed prototype cellular phone terminal is described in the following. Fig. 4 shows an outline of the prototype, while the principal system parameters are summarized in Table 1. To achieve high performance, SH-Mobile V [3] CPU is adopted. The terminal is equipped with a DTV receiver module that includes the RF antenna and the DTV tuner for the OFDM. The transport stream format and coding syntax of each medium strictly conform to the Japanese DTV standard specified in ARIB STD-B24 [4]. Fig. 5 shows the configuration of software modules embedded on the hardware described above. The meshed region in the figure corresponds to specific software modules for receiving DTV programs. The TS controller module controls each media decoder module to maintain the media synchronization. The application manager provides a user interface and controls subsidiary modules. A browser module is com- KOIKE et al.: PERSONAL MOBILE DTV CELLULAR PHONE TERMINAL DEVELOPED FOR DIGITAL TERRESTRIAL BROADCASTING 283

Fig. 4. Outline of prototype terminal. Table 1 System Parameters of Prototype Terminal monly used for DTV BML contents and Internet Web contents. A. Application Manager Module An application manager module controls all software tasks including low-level software engines. This main module is comprised of the tasks listed below. 1) TS Packet Manager: The TS packet manager task provides the packet scheduling mechanism for the TS control module in order to eliminate the network jitter from a received transport stream. This task also provides a buffering mechanism for Internet web content. 2) GUI Control: The GUI control task provides a GUI framework and manages user events and application status. 3) DSM-CC Manager: The DSM-CC manager task manages BML data objects according to instructions from a DSM-CC carousel decoder module. The main function is a content cache and a data delivery for the XHTML browser module by which the browsing performance improvement can be achieved. 4) Emergency Receiver: To realize the emergency warning system, a trigger message is emitted for selected terminals. An emergency receiver task receives and interprets the trigger message, and activates the appropriate DTV function. B. TS Control Module The TS control module provides MPEG-2 TS and MPEG-2 packetized elementary stream (PES) protocol stacks, and maintains media synchronization of real time contents (audio and video streams). Fig. 6 shows the employed synchronization mechanism. and indicate the buffer occupancy levels with regard to the audio presentation unit (PU) and video PU, respectively. The presentation timing for an audio frame and a video frame is managed in the following way. 284 PROCEEDINGS OF THE IEEE, VOL. 94, NO. 1, JANUARY 2006

Fig. 5. Software configuration embedded on prototype terminal. 1) Audio Stream: Reconstructed digital audio data are stored in an audio PU buffer. When the occupancy level reaches the threshold value TH, audio play is started and the current local system time clock (STC) time is set at the same value as the first audio presentation time stamp (PTS) value PTS. Then, STC time is adjusted based on the system clock frequency (27 MHz) calculated from the received program clock reference (PCR) value. 2) Video Stream: Video frame output is conducted by directly refreshing the VRAM when corresponding to PTS time. As long as the audio initialization process has not established local STC time, received video PES packets are discarded. C. Demuxer Module The demuxer module extracts media elementary streams (MPEG-4 Visual, MPEG-2 AAC and DSM-CC) from a transport stream received from TS control module. D. MPEG-2 AAC Decoder Module The MPEG-2 AAC decoder module reconstructs digital audio data from the MPEG-2 AAC elementary stream. E. MPEG-4 Visual Decoder Module The MPEG-4 visual decoder module reconstructs image data from the MPEG-4 visual elementary stream. F. DSM-CC Carousel Decoder Module The DSM-CC carousel decoder module extracts BML objects from the DSM-CC elementary stream and notifies the application manager of an event whereby new BML objects have been detected. G. XHTML Basic Browser Module The XHTML Basic browser module presents XHTML contents received from the application manager. This module is composed of XHTML1.1/Basic profile, CSS2 subset, and ECMA scripts. This browser conforms to the basic BML module specified in ARIB STD-B24 [3] exactly. H. TCP (UDP)/IP Control Module The TCP (UDP)/IP control module provides TCP/IP and UDP/IP protocol stacks on the network interface (CDMA 1X WIN), while extracted payload data units are delivered to the application manager module. Fig. 7 shows the receiving process for Internet streaming from the program distribution. DTV contents in MPEG-2 TS (simply called TS in the following) format are transmitted over Real-Time Transport Protocol (RTP). This protocol is a representative real-time streaming protocol over the UDP/IP protocol [6], [7]. Table 2 summarizes several parameters specifying the receiver process of streaming. can be calculated by (1) When packet loss occurs, no TS packets exist in the TS buffer even at the corresponding scheduled timing according to. In this case, a periodic input process for the TS (1) KOIKE et al.: PERSONAL MOBILE DTV CELLULAR PHONE TERMINAL DEVELOPED FOR DIGITAL TERRESTRIAL BROADCASTING 285

Fig. 6. Synchronization mechanism for audio and video presentation units. Table 2 Operation Parameters of Streaming Client Fig. 7. Receiver process for Internet streaming. decoder is maintained by feeding the dummy data whose length corresponds to the lost data. V. PERFORMANCE EVALUATION For the performance evaluation of the developed terminal, several experiments were conducted by using an actual terminal and an in-service cellular phone network. From the experimental results, the total software performance of DTV client was first evaluated, and then the performance, especially when operating as the RTP streaming client, was evaluated. A. Software Performance The software performance was evaluated under service parameters particularly for terminal operations on the TV play mode as shown in Table 3. ISDB-T means the digital terrestrial broadcasting scheme adopted in Japan [5]. Active vertical size in the video coding is smaller than the value in the table so that the aspect ratio in active video samples can be kept at 16 : 9. As experimental results, the CPU usage ratio of each internal task is summarized in Table 4. In order to obtain detailed results precisely, the experiments were conducted by 286 PROCEEDINGS OF THE IEEE, VOL. 94, NO. 1, JANUARY 2006

Table 3 Operation Parameters for Performance Evaluation Table 6 Condition of Transmission Experiment Table 7 Initial Buffering Level Required to Prevent TS Buffer Underflow Table 4 CPU Usage Ratio of Each Internal Task Table 5 Definition of Software Module Category from the program distribution server was observed. Table 6 shows the transmission condition. As for the mapping mechanism of TS packets into an RTP packet, was selected as an integer whose value is from three to seven so that the IP packet payload length could be kept smaller than 1500 B, accounting for preventing IP packet fragmentation. At the program distribution server, the timing for sending every RTP packet is controlled with sufficient precision to keep the interval of successive RTP packet transmission at a close value to calculated by (2) (2) using the prototype terminal formerly developed [2]. Evaluated software modules are categorized as shown in Table 5, and the CPU usage ratio is represented by a percentage. From these results, it was confirmed that the occupancy ratio using a system control task increased when initializing the receiving process of DTV programs. The result only includes the averaged value, while the result for the A/V decoder task was confirmed to have a fluctuation that depends on image characteristics such as the distribution of moving vectors. In addition, the occupancy by XHTML browser task was concentrated only for a relatively short period while BML or XHTML data objects were updated. B. Performance of Internet Streaming In order to evaluate the performance of TCP (UDP)/IP controller, packet receiving process for Internet streaming First, the arrival time of every RTP packet was observed at the terminal to evaluate the relation between the receiver performance and the RTP payload length. The arrival time is expressed by the number of elapsed seconds after the arrival of the first RTP packet. The TS buffer occupancy level is calculated by (3) where indicates the total number of received RTP packets at the time. From the result of, the initial buffering level required to prevent the TS buffer underflow was estimated. Table 7 shows the estimated initial buffering level. was set identical to in order to simplify the condition. Fig. 8 shows the buffer occupancy level observed under (, ). There was no IP packet loss detected in those experiments. (3) KOIKE et al.: PERSONAL MOBILE DTV CELLULAR PHONE TERMINAL DEVELOPED FOR DIGITAL TERRESTRIAL BROADCASTING 287

[3] News releases from headquarters, [Online]. Available: http://www.hitachi.com/new/cnews/e/2002/0930/index.html [4] Data Coding and Transmission Specification for Digital Broadcasting, ARIB STD-B24, Association of Radio Industries and Businesses, Feb. 2003. [5] Error-correction, data framing, modulation and emission methods for digital terrestrial television broadcasting, Recommendation ITU-R BT.1306-1 (draft revision), Jul. 2005. [6] RTP: A transport protocol for real-time applications, IETF RFC 3550, Jul. 2003 [Online]. Available: http://www.ietf.org [7] RTP payload format for MPEG1/MPEG2 video, IETF RFC 2250, Jan. 1998 [Online]. Available: http://www.ietf.org Fig. 8. TS buffer occupancy L(t) (NT =7,LS =1). From the results above, it was confirmed that the required initial buffering level was almost equivalent among evaluated parameters for. With regard to the CPU load caused by the TCP (UDP)/IP manager, the frequency of RTP packet receiving process can be decreased as the RTP payload length becomes larger. Therefore, it can be concluded that the optimal value for is given by the largest value that meets the limitation for preventing the IP fragmentation. information system. Atsushi Koike was born in 1961. He received the M.E. degree from Tohoku University in 1985 and the Ph.D. degree from Kyoto University in 2002. He has been working at KDDI, Saitama, Japan, since 1985. He is now the group leader of Visual Communications Laboratory, KDDI R&D Laboratories Inc. He has been engaged in research and development on visual communication technology such as high efficiency coding of moving images, model based image coding, and still image transmission systems, and medical VI. CONCLUSION This paper describes an assumed DTV service (so-called integrated service) for personal mobile terminals, and the technical architecture of the developed terminal with regard to the hardware and the software. We also conducted several experiments using the developed prototype terminal. From these results, the total software performance on the typical DTV operation and the receiver performance for Internet streaming from the program distribution server were evaluated. Future study includes a field trial in which the developed mobile terminal receives the DVT program transmitted from an actual broadcasting station. ACKNOWLEDGMENT The authors would like to thank the Central Research Laboratory of Hitachi Ltd., Japan, for much cooperation in developing the mobile DTV cellular phone terminal. REFERENCES [1] H. Kokubun et al., Development of a Simulator for Integrated Internet and Broadcasting Services, ITE Tech. Rep. vol. 26, no. 24, pp. 37 42, Mar. 2002. [2] S. Naito et al., Personal mobile DTV terminal designed for service trial of digital terrestrial broadcasting with full use of Internet connectivity, in Proc. ICMU 2004 pp. 198 204. Shuichi Matsumoto was born in 1954. He received the M.E. and Dr.Eng. degrees from Hokkaido University in 1979 and 1990, respectively. He has been working at KDDI, Saitama, Japan, since 1979. He is currently the executive director of KDDI R&D Laboratories Inc. He has been engaged in research and development on video transmission technology such as high efficiency compression coding of moving pictures especially for SDTV and HDTV. Hideki Kokubun was born in 1950. He received the B.S. degree from Tokyo Institute of Technology in 1974. He joined NHK (Japan Broadcasting Corporation), Tokyo, in 1974. Since 1978, he has been working at the Science and Technical Research Laboratories (STRL) of NHK and has been engaged in the research and development of very large scale integration (VLSI) for high-speed image processing, especially in the development of HDTV decoders and PDP signal processors. He joined NHK Engineering Services Inc. in 2001. He has been engaged in the research of the advanced information services research laboratory. He has been engaged in the research and development of a useful mobile receiver and new network-linked broadcasting services. He is currently a Senior Member of the research staff in Networked Broadcasting Systems, STRL. Mr. Kokubun is a member of the Image Information and Television Engineers of Japan and the Institute of Electronics, Information and Communication Engineers of Japan. 288 PROCEEDINGS OF THE IEEE, VOL. 94, NO. 1, JANUARY 2006