PRODUCTION OF TV PROGRAMS ON A SINGLE DESKTOP PC -SPECIAL SCRIPTING LANGUAGE TVML GENERATES LOW-COST TV PROGRAMS- Douke Mamoru Ariyasu Kyoko Hamaguchi Narichika Hayashi Masaki Japan Broadcasting Corporation (NHK), Japan ABSTRACT We have developed a system for automatically generating TV programs based on scripts written on a desktop PC. The scripting language, which we call TVML (TV program Making Language), is interpreted by software running on a Windows PC called TVML Player to produce a full-featured TV program. TVML Player generates programs in real time using real-time computer graphics, voice synthesis, and other techniques. We have also developed a new tool called TVML Author that enables anyone to produce their own TVML-based shows. It includes a WYSIWYG (what you see is what you get) capability that enables users to create programs merely by manipulating the graphical user interface. This paper describes the overview and the key features of TVML, and also introduces the capabilities of TVML Author. The abilities and power of the TVML are amply demonstrated through actual examples of TV content created using TVML. 1. INTRODUCTION These days, the number of media and the required amount of content have been increasing sharply, but, on the content production side, there is no hope for any dramatic increase in production facilities, human resources or budget. With limited resources and budget, we are faced with an inescapable need for program production methods that are far more efficient than conventional methods for content creation. As a means of solving this problem, technology for the automatic generation of content by computer and the processing of the generated content (which we call content handling technology) has become extremely important. We have proposed a TV program description language, TVML (TV program Making Language) [1][2], as technology that implements content handling, mainly for TV program production, and we are proceeding with research and development of that technology. TVML is a special language for using computers to describe TV programs. When a program script written in this language is input to software called the TVML Player, a TV program is generated using real-time CG, voice synthesis, and other such computer multimedia capabilities. Here, we describe TVML and introduce TVML Author, a user-friendly authoring tool for simple production of programs with TVML. This software allows even someone who has no knowledge of TVML to create programs simply by manipulating the GUI (Graphical User Interface) on the computer while actually seeing the video that is generated. We also present an actual example of using TVML
to create a program and describe examples of several studies concerning the generation of content based on TVML. 2. THE TVML PROGRAM DESCRIPTION LANGUAGE 2.1 What Is TVML? TVML is a kind of programming language that can be used to describe a television program completely. The description style of TVML is based on the method of writing the program scripts that are used in actual television program production. It has extremely high readability by humans and is designed to be easy to use even for those who might be called amateurs at using a computer. TVML provides the functions that are needed to create ordinary information programs in the form of commands (about 90 commands). These types of information programs, which include news, event guides and so on, are in the genre of programs that present certain kinds of information. 2.2 The TVML Language Specifications The TVML commands are used to move computer graphic (CG) characters, control the CG camera, play movies and so on, and are provided for each program element. All of the commands have the same format, which is shown below. Event type: command name (arg1=data1, arg2=data2, arg3=data3,...) The event type indicates the object of the command, such as character, camera, etc. The command name specifies the action that affects the object of the command. The parameters that are required for each command name are provided as arguments enclosed in parentheses. Because all of the commands have the same format, even users that are not familiar with TVML can easily grasp the general idea of an action and can quickly learn TVML itself. Also, script writing with TVML proceeds in order from the beginning of the program to the end, as it does in an actual program script. Program scripts written with TVML consist of multiple lines of TVML script, with each line corresponding to one event (action) in the program. An example script is shown below. character: bow(name=bob,wait=no) character: bow(name=mary) character: talk( name = BOB, text = "Hello everybody!" ) When this script is input into the TVML Player, a scene results in which the two CG characters named BOB and MARY first bow at the same time and then BOB says, "Hello everybody!". Some examples from the TVML command set are presented in Table 1. 2.3 The TVML Player The TVML Player is a software program that runs on a personal computer. The TVML Player interprets and executes scripts written in TVML line by line and outputs the TV program in real time. The TVML Player generates studio shots with real-time computer graphics, and CG characters that appear in a CG studio set speak the spoken parts that are described in the TVML script in synthesized
voices as the action is shot by the camera. In addition to that, video playback, superimposing of text and image titles, playback of audio data files for background music, narration by voice synthesis and other such effects can be generated in real time to create audio-visual television programs. The TVML Player has two operation modes: the normal mode and the control mode. In the normal mode, the TVML script for one program is input, the script is interpreted one line at a time from the beginning to the end, and the program is generated. This mode is used to playback ordinary programs. In the control mode, on the other hand, as shown in Fig. 1, data can be exchanged with external software via shared memory, allowing the TVML Player to be controlled from the external software by sending TVML script or controlling commands to the TVML Player one line at a time. The control mode is explained in more detail below. The TVML Player operates entirely as an interpreter, so when one line of script is read in, it is parsed and executed immediately, and when the execution is completed, the next line is read in. Accordingly, when the TVML Player is running in control mode, script sent to the TVML Player asynchronously from an external program is executed by the TVML Player immediately, and then the TVML Player waits for the next input. Using this function, it is possible to develop interactive applications with TVML by writing separate software that allows interaction with users by sending scripts and commands to the TVML Player and receiving status information in return, as shown in Fig. 1. Function Table 1. Examples of TVML commands Figure 1: TVML Player external control mode 3. THE TVML AUTHOR The TVML Author is a user-friendly GUI based authoring tool that we have newly developed for creating programs with TVML. Until this tool became available, the user had to understand the TVML language specifications and use a text editor to write TVML script. Although, as mentioned above, the TVML language specifications have extremely high readability, the appropriate parameters for setting the camera size and so on must be selected by a repeated process of entering the parameters and then running the script to see the effect. TVML Author eliminates that trouble by providing a graphical user interface (GUI) that allows the user to interactively select the materials that are used in creating the
TVML programs and perform the entire script writing process smoothly, all without knowledge of TVML. Because the programs created with TVML Author are automatically converted to TVML scripts, they can be played back on the TVML Player. The GUI windows of the TVML Author are shown in Fig. 2. In the upper left is a preview window for the program that is being created; in the lower right is the spreadsheet for creating the program, in the upper right is a window for setting the parameters of the events that are selected in the spreadsheet. A table-type spreadsheet like Microsoft Excel is used. One cell corresponds to one TVML event (one line), and the events proceed from top to bottom. Events that are in the same row are executed at the same time. Each cell can be freely cut and pasted or moved. Figure 2: The TVML Author windows Figure 3: The direct script editing window When a cell is selected, the parameter set for that event is displayed in the upper right-hand window, and the values of those parameters can be set simply by using the mouse or keyboard. In particular, the parameters that require visual confirmation, such as the position of the CG characters and the camera angle and position, can be set interactively by dragging the mouse while observing the effect in the preview window. TVML Author can also be switched at any time to the spreadsheet editing mode, as shown in Fig. 2, or the direct TVML script editing mode, as shown in Fig. 3, to allow whatever editing style the user prefers. The TVML Author also has abundant material setting functions, so that CG characters, CG studio sets, CG props, movies, music and other such elements can be easily set and managed individually by using the GUI, as shown by the example in Fig. 4. In addition to those editing modes, there is also a function for displaying and editing only the spoken words that are contained in the script. (See Fig. 5.) This function can be used to edit only the spoken dialog in the program, while also allowing text to be copied and pasted from a word processor or other such source. Because the TVML scripts of programs that have been created with TVML Author include all of the configuration information, such as camerawork, CG character positions, set arrangements and other such information, they can also be used as a kind of template for creating serial programs (language study programs, etc.) which have the same program configuration, but present different content, by simply replacing the picture and sound materials and the spoken content. Using TVML Author in this way makes it possible to create programs in a short time and with greater efficiency.
An example of the output of a program created with TVML Author is shown in Fig. 6. The TVML Author is simply a TVML authoring tool. It is also possible to create the same kinds of programs without a TVML authoring tool by writing TVML script directly with a text editor, but using the TVML Author makes it possible to create these kinds of programs in a very short time. Figure 4: The TVML Author CG character settings window Figure 5: TVML Author dialog editing window Figure 6: Example of a program made with TVML Author 4. APPLICATION OF TVML TO THE CREATION OF ACTUAL CONTENT As we have described so far, television programs can be created simply, at low cost and in a short time by using TVML. Examples of programs that have been used in actual broadcasts are described below. - Variety program (aired on NHK) As part of a variety program, a CG character sketch made using TVML, part of an information program made with TVML, and a TVML program that was generated in real time were aired. Those programs could be created with TVML at extremely low cost and in a short time. - Weather report programs (aired on local NHK stations) Part of a daily weather report program was created using TVML, featuring a scene with excellent
visual effects achieved with real-time 3D computer graphics. This program is still being broadcast at this time. - Commercial programs (aired by the Akashi CATV Company) This cable TV company procured a personal computer for creating TVML programs. They are introducing TVML CG characters one by one to sketches in local weekly programming. The programs are created simply by changing only the spoken content of each week. These programs are also currently being broadcast. A number of broadcast stations other than those listed above are working on the creation of information programs with TVML. Other than the production of television programs, TVML can also serve as a simple yet powerful tool for the production of demonstration videos and educational videos. Examples are presented below. - Demonstration videos At a BS digital broadcasting exhibition site, TVML is being used to present an automated explanation of what BS digital broadcasting is and what special features it has. Because the explanation is presented by CG characters in a comical manner, it is well received by visitors. - Educational videos Content created with TVML is being used as part of a university computer programming language course. The students are able to learn about programming languages on their own by viewing programs described in TVML. 5. RESEARCH ON CONTENT HANDLING METHODS BASED ON TVML In addition to being used as a tool for creating content in a television program format, TVML can also be used as listed below - It is useful as a television program style presentation engine. - Because TVML is positioned as an intermediate between scripts written in natural language and content generated by means of the multimedia functions of computers, content can be created automatically by the automatic generation of TVML script. - Real-time interaction is possible by using the control mode of the TVML Player. By making use of these special features, TVML can be applied to various content handling methods. Below, we briefly introduce some examples of research on content handling methods that are based on TVML. (1) Automatic production of individualized programs [3] By preparing multiple TVML scripts to serve as templates and inputting the core data to those templates, TVML scripts can be generated automatically. This processing can be done on a single desk-top personal computer, so programs that suit the individual preferences of a particular person can be created automatically on a personal computer. (2) Automatic generation of conversation programs from Web pages [4] Technology for automatically or semi-automatically converting the content of different media is called media conversion technology. One example is a method for automatically converting interviews from a Web page to a talk show format. The HTML file is analyzed to extract the title, name of the interviewed guest, and the dialogue. That information is automatically converted to a TVML script for a dialogue between two CG characters in order to generate a talk show. The camera switching and gestures that are important in the performance of talk show are done automatically
using statistical data on camera switching from actual talk programs, information on the speakers, and timers. (3) Interactive content Using the external control mode of the TVML Player, it is possible to generate TVML script by using external software and then send that script to the TVML Player asynchronously. Thus, quiz programs, query type manuals, and multi-story programs can be created easily. 6. CONCLUSION We have introduced TVML and described how TVML can be used as a television program production tool and as an infrastructure for creating new types of content. Concerning the former use, we described TVML Author and presented examples of its actual use at broadcasting sites. For the latter use, we introduced examples of research using TVML. These examples demonstrate that, as an infrastructure, TVML has potential for various types of development in the fields of TV program production and content handling. TVML Player is being distributed as freeware, and it had been downloaded about 4000 times at the time of this writing. By providing software that can be used to create television programs on a personal computer, we intend to release this capability not only to some professionals, but to amateurs as well. In this way we intend to contribute to the relief of the insufficiency in content by a bottom-up increase in the number of program creators. Furthermore, because TVML can be used to produce programs quickly and at low cost, it is also a very capable tool for professional program directors. It can be said to be particularly promising as a program production tool for local stations, small video productions which have limited production budgets and time, and for Internet broadcasting stations. For more information on TVML or to download the program, please go to the TVML Web site at the following URL. http://www.strl.nhk.or.jp/tvml/ REFERENCES 1. Hayashi, M., Ueda, H. and Kurihara, T., TVML (TV program Making Language) Automatic TV Program Generation from Text-based Script, Proceedings of Imagina 99, (1999) 2. Hayashi, M., Douke, M., Ariyasu, K., Hamaguchi, N., TV Program Making Language (TVML) Benefits of Object-based Contents-creation Method-, IBC2001 Conference Publication Volume2 pp.23 to 31, (2001) 3. Douke, M., Hayashi, M., Makino, E., A Study of Automatic Program Production Using TVML, Short Papers and Demos, Eurographics 99 pp.42 to 45, (1999) 4. Hayashi, M., Gakumazawa, Y., Yamanouchi, Y., Automatic Generation of Talk Show from Dialog using TVML, ICCC 99 Digital Convergence for Creative Divergence Vol.2 pp. 325 to 332, (1999)