Overview of Information Presentation Technologies for Visually Impaired and Applications in Broadcasting

Overview of Information Presentation Technologies for Visually Impaired and Applications in Broadcasting It has been over 60 years since television broadcasting began in Japan. Today, digital broadcasts provide various data besides video and audio, and technologies such as the Internet and smartphones have spread almost everywhere. Users can easily obtain and produce information, and the circulated information is diverse in content. Although these new technologies have enriched the lives of many people, most of the information they convey is only presented visually, in the form of video data. Video data is excellent for presenting information at a glance and changes with time, but our reliance on it can be a major barrier for people with visual impairment. Is it possible to improve the ways in which we convey the content of images to people with visual impairment? Research is being done on technologies that can lower this barrier, and little by little, the barrier is falling as they are made available. In this special feature, we introduce some of the information presentation technologies for visually impaired that are currently being researched, including a high-speed speech conversion technology, and haptic displays. Before doing that, however, this article will give an overview of barrier-free technologies for visually impaired that have been implemented in television broadcasting and touch on new technologies to convey the shape and hardness of 3D objects through the sense of touch. 1. Introduction According to a study of youth and adults with physical disabilities conducted by the Ministry of Health, Labour and Welfare in 2006, there are approximately 310,000 people with visual impairment in Japan *1. Among these, 90,000 are *1 A survey of youth and adults with physical disabilities was conducted in FY2011, but due to the Tohoku Earthquake, only part of the survey was completed. The official results were not published. totally blind, and approximately 220,000 have some level of impairment (poor vision) 1). Some people have been issued with physical disability certificates, but there are likely considerably more people with poor vision who have not applied for or received a certificate. In fact, according to a survey by the Japanese Ophthalmological Society, based on standards of visual impairment used in the USA 2), it was estimated there are approximately 1.64 million people with visual impairment in Japan as of 2007; this number is substantially more than the number of issued physical disability certificates. Also, the above numbers do not include people with special needs or disabilities other than those affecting visual acuity or field of view, which is defined in terms of level of acuity and field of view in vision tests done after any correction or treatment of impairment. Such impairments include those unique weak light sensitivity impairing light perception *2, refraction *3, color, motion, or accommodation. Regarding color perception, approximately 5% of men and 0.2% of women in Japan have color vision deficiency and have difficulty distinguishing among certain combinations of colors. The Ministry of Internal Affairs and Communications (MIC) as of February, 2015 estimates that 3.05 million men and 130,000 women are affected. Most of these cases are hereditary, difficult to correct, and create inconvenience in the daily lives of those affected. Another impairment, resbyopia manifests itself as a weakening of motor and accommodation function, and it affects most elderly people 65 years of age and older, which is one in four, or 33.189 million, people in Japan. These people are not included in numbers of the visually *2 Many people experience visual impairments such as nightblindness, day-blindness, and photophobia, in which people experience abnormal discomfort or pain due to ordinary levels of light. *3 Impairments related to refraction, such as double vision. 2

Feature impaired either. Thus, there are many people besides those who are blind or have poor vision who have difficulty getting the information they need from television. We believe barrier-free technologies must be provided for these people. This article describes an example of such a technology that presents tsunami bulletins and warnings using colors which are easier to distinguish by people with color vision deficiency. Our research on effective presentation of television video content to those with visual impairment or those who experience difficulty acquiring information from video is somewhat a trial-and-error effort. However, television is not simply a tool for conveying information; it is also a shared social resource that stimulates communication between people through the topics it presents, and we are conducting our research premised on the belief that TV should build understanding among people and enrich their lives. This article introduces various initiatives in the television broadcasting world that are intended to visually impaired, including described video, technologies that read out captions, improvements to color representation of tsunami bulletins and warnings, and a technology that conveys the shape and hardness of 3D objects through the sense of touch. 2. Barrier-free technologies for visually impaired A survey conducted during the analog era of television 4) reported most blind and poor-vision respondents named television rather than radio as their most-used means of obtaining information, and even totally blind respondents said they depended on the information in television news audio. Back then, some FM radio receivers were able to receive audio from VHF channels 1 to 3, and many people with poor vision living in urban areas preferred to use such receivers. However, since the switch over to digital terrestrial broadcasting, which uses UHF channels that are more easily obstructed by buildings and other structures, more people are unable to receive signals indoors or cannot get stable reception. Users have also had to switch to receivers that can receive One-Seg audio, and these require one to navigate a multi-level menu, which is difficult for users with visual impairment. Thus, compared with analog TV and radio, which only required one to switch the receiver on and dial or press buttons to find a station, digital television is a little less user-friendly for blind and poor-vision users. These days, personal computers can read out text and information from the Internet, and many users with visual impairment use the Internet to obtain information. There is also software that adjusts colors to make them more visible to those with color perception impairments. Such software is being used by government and other web sites to enable users to change the normal colors of the displayed information to ones they find easier to see. However, although such websites are useful, television by its nature is still an important source of information for visually impaired as it sends information many people simultaneously and conveys the latest information during emergencies. This article introduces the latest technologies and research currently in progress to realize barrier-free television for users with visual impairment. 2.1 Described video Video information on the television screen can be converted into an audio and presented to users with visual impairment. A typical example is descriptive audio broadcasts or described video, in which the content of the video is described on the secondary audio channel. It was first used by a commercial broadcaster in 1983 and has been used by NHK since 1990 (NHK first used it in the morning serial Rin-rin to ). These early described video services included descriptions of the cast of the program, the appearances and actions of the characters, and explanations of the background and scenario. Script writers and directors had to select the information to be conveyed during editing, and only a limited amount of information could be added in this way. The timing of the scripted events and background music also had to be considered, so the narrators of the description had to be very skillful. Most described video television broadcasts provide a good level of description, comparable to radio dramas that use only audio, and are evaluated highly by visually impaired listeners, but their production requires numerous specialists, as described above. As personnel and time are limited, it is not used for many programs. Outside of Japan, in the USA for example, the Public Broadcasting Service (PBS) began described 3

Broadcaster Table 1: Record of described video broadcasts by NHK and Private broadcasters in Tokyo (FY2013) Percent of broadcast programs with Percent of total broadcast time with described video that are subject to policy described video targets NHK (Educational) NHK (General) TV Asahi Corp. Fuji Television Network Inc. Nippon Television Network Corp. Tokyo Broadcasting System Holdings Inc. TV Tokyo Corp. 13.6 9.8 9.3 5.7 4.3 4.2 4.1 12.0 8.9 3.9 2.2 1.3 1.4 1.8 video broadcasts using the secondary audio channel at the same time as NHK, in 1990. Yet there remain many outstanding issues preventing the inclusion of described video in most broadcasts. Table 1 shows excerpts from the MIC publication, 2013 Captioned broadcast performance listing the amount of described video at NHK and other commercial broadcasters in Tokyo 5)*4. As the table indicates, rates are not high, despite a strong demand from visually impaired to increase the number of such programs *5. Furthermore, there is demand for described video containing faster speech that provides more information, even if it affects the quality of the production. In this regard, balancing the level of production perceived by viewers and the information conveyed has been a persistent problem, so consideration must also be given to preserving and improving the quality of content rather than simply increasing the number of programs. These two desires of visually impaired suggest that we should develop described video with faster speech and more information. As such, we are researching a production support system for described video broadcasts that uses speech synthesis and allows users to adjust the speed at which the content is read out to their preference. *4 The programs represented in Table 1 include all programs broadcast between 7 am and midnight, except for those to which described video could not be applied because of property rights or other restrictions. This includes programs restricted due to property rights, as well as broadcasts that already use two audio channels for a secondary sound track or second language, 5.1-channel surround sound broadcasts, and programs that have no gaps in which to augment the main soundtrack. *5 Similarly, viewers with visually impairment have requested described video services for live broadcasts; the Paralympic games coverage in 2020 will include described video. 2.2 Speech synthesis of broadcast information Digital broadcasting enables viewers to access news and weather forecasts at any time as well as an Electronic Program Guide (EPG) showing programming for the coming week. These data broadcasts and the EPG store and update the latest information in the memory of the television and display it on the screen when viewers want to use it. The information covers many categories such as news, weather forecasts, and sports. However, users must use menus to reach the information they are interested in and no guidance is available for users with visual impairment. To eliminate this barrier, we have developed a digital broadcast receiver function called Visually Impaired Assist TV (VIA-TV), which presents data broadcasts and EPG content using speech. VIA-TV has a function that replaces text information with speech, and it can also display text information in Braille when operated on commercially available Braille displays. It is also able to display data broadcast menus on 2D haptic displays that support Braille, so visually impaired can use their fingers to understand the number and relative positions of items on the data broadcast menu. The EPG displays the program guide on the television screen, but the information that viewers use to decide what to watch, such as start and end times, title, subtitles, subject, and actors, are all displayed as text. In data broadcasts, the numbers and positions of items are determined by the broadcasters, so it is difficult for a general purpose device to read them out. However, since the EPG is regulated to be consistent, some receiver makers have made this possible *6. *6 As of July, 2015, TV receivers and receivers with recording functions from Mitsubishi and Panasonic, and recorders from Panasonic have the functions to read out the EPG. 4

2.3 Improving the color of diagrams expressing warnings It is estimated that more than three million people in Japan, mostly men, have difficulty distinguishing colors. The cells in the human eye that perceive color are called cone cells, and there are three types that each respond mainly to one of the primary colors, red, blue or green. The human brain distinguishes colors using the signals from these three types of cells, but if some of them have low sensitivity, or if the sensitivity peak is different, idiosyncrasies in sensitivity to color can result. Color perception that is different than usual can be categorized into four types: protanopia (P-type), or low sensitivity to red, deuteranopia (D-type), or low sensitivity to green, tritanopia (T-type), or low sensitivity to blue, and achromatopsia (A-type), or sensitivity to only one color. Most people that do not have normal color perception have P-type or D-type, or T-type; A-type is rare. The degree of P or D-type varies by individual, but those with strong P or D-type vision may perceive green and red as the same color. Figure 1 simulates what our laboratory mascot, Labo-chan, would look like to people with different types of color vision. Figure 1 (a) is the original image, while (b) and (c) have been processed to simulate how they would look to people with P-type and D-type vision. An easy-to-understand way of indicating colors is to present their names next to them. One example of this naming scheme is the four color buttons of a digital television remote control. The trouble is that this method is useful for a limited number of typical colors; more characters are required to represent the whole spectrum of color, and this means a screen would sometimes be filled with the names of different colors. The usefulness of this method seems to be limited. As broadcasters, we must consider people that do not have typical color perception when expressing important information using colors. Tsunami bulletins and warnings are one such example. These emergency bulletins and warnings express their information in an easy-to-understand way by coloring areas on a map of Japan. Although the names of the affected coastal areas are listed on screen and read out by the announcers, the map is a good way of providing a summary of the information at-a-glance. Tsunami information includes three levels: tsunami bulletin, (a) Original image (b) P-type (intensity) Difficulty distinguishing red (c) D-type (intensity) Difficulty distinguishing green Figure 1: Simulation of P-type and D-type color appearance tsunami warning, and large tsunami warning, and in some cases, all levels could be displayed at the same time. In the past, each broadcaster had its own way of displaying these bulletins and warnings. For example, for each broadcaster would use different sets of colors to show the land and sea areas on the map and would highlight the coastal areas subject to the bulletins and warnings by overlaying lines of different thicknesses and colors. Changing channels would thus change the display style, making it difficult to understand the information correctly 5

Before improvement Red-white double line Large tsunami warning Red Yellow Tsunami warning Tsunami bulletin After improvement Purple thick line Large tsunami warning Red Yellow Tsunami warning Tsunami bulletin Figure 2: Use of color in tsunami bulletins and warnings (before and after improvements) without supplementary textual explanations. Then in 2010, prompted by a tsunami from an earthquake in Chile, NHK and other private TV stations in Tokyo decided to unify their display styles; they also selected color combinations that would help people with different sorts of color perception distinguish the bulletins and warnings more easily. Figure 2 shows the screens before and after these improvements. This system began operation in August, 2011 *7. *7 The Tohoku Earthquake occurred just prior to the planned unification in March, 2011 and delayed its deployment. 2.4 Technologies that convey the shape and hardness of 3D objects to hands and fingers It is difficult to convey 3D information such as the shape of solid objects on 2D displays such as VIA- TV. Furthermore, although images can be described in words, doing so would take a long time and the viewer would still have difficulty understanding the information. It is hard to use words to express the hardness of an object because the user needs to have experienced that object and remember the experience. Our laboratory is conducting research on technologies that enable visually impaired to perceive information such as shape and hardness. One such technology provides multiple stimuli to a single finger to represent properties such as color. 6

Feature 3. Conclusion Tokyo 2020 Olympic and Paralympic Games will be an opportunity to showcase 4K and 8K television broadcasting. Individual viewers will no doubt understand and form opinions on the programming in their own way, and they will be able to share their opinions and feelings about the competition and have a chance perhaps to laugh or cry together over the results. For this to be possible, content has to be delivered to many people simultaneously by radio and television. Television has not simply gone through repeated increases in image quality, resolution and screen size, it has become an extremely convenient service thanks to major innovations. The change from analog to digital has made it easier to caption broadcasts and show text of what is being discussed in the program; data broadcasts enable viewing of news and weather information at any time; and the EPG enables times to be reserved on the screen for viewing or recording programs. R&D is progressing on new services that take advantage of the improved image quality, larger screen size, and higher sound quality of 4K and 8K. How can televisions be made to have the same levels of convenience, familiarity, and utility for everyone? We will continue to seek answers to this question and develop technologies that reduce disparity in information available to people with and without visual impairments. (Satoru Kondo) References 1) Ministry of Health, Labour and Welfare: 2006 Study of youth and adults with physical disabilities-- Results, http://www.mhlw.go.jp/toukei/saikin/hw/ shintai/06/index.html (in Japanese) 2) Japanese Ophthalmological Society Press Release: Social cost of visual impairments: 8.8 trillion yen!! First ever trial calculation of decreases in productivity and QOL due to visual impairments, http://www.gankaikai.or.jp/press/20091115_ socialcost.pdf (in Japanese) 3) Ministry of Internal Affairs and Communications, Statistics Japan: Population Estimates as of July 1st, 2015, http://www.stat.go.jp/data/jinsui/pdf/201507. pdf (in Japanese) 4) Japan Federation of the Blind et al.: Report on study and development status of described video (secondary audio) for the person with visual impairment: Creating universal programs friendly to all - FY2004 (2005) (in Japanese) 5) Ministry of Internal Affairs and Communications Report: FY2013 Caption broadcast performance, Appendix 3, Described video broadcast performance by broadcaster, http://www.soumu.go.jp/main_ content/000317168.pdf (in Japanese) 7