National Institute of Information and Communications Technology, Japan

Size: px

Start display at page:

Download "National Institute of Information and Communications Technology, Japan"

Asher Little
5 years ago
Views:

1 PCS2010 Research on Ultra-Realistic Communications December, 2010 Kazumasa Enami Vice President National Institute of Information and Communications Technology Overview of NICT Outline Towards Ultra-Realistic Broadcasting and Communications Ultra- (Super Hi-vision) 3DTV Multi-sensory Interaction System Data-rate of Future Video Systems Summary 2 Overview of NICT National Institute of Information and Communications Technology, Japan Outline of NICT Main operations (Incorporated Administrative Agency, NICT Act) R&D of technology relating to information and communications technology (ICT) and radio waves Support for those conducting R&D in advanced telecommunication and broadcasting Promotion of enterprises within the telecommunication and broadcasting area Budget & Personnel (FY2010) Budget : billion yen Personnel: 873 (Total Researchers: 545, PhDs : 398) 3 4 1

アジア ( シンカホールタイ ) へ米国へ The NICT was born by merging CRL and TAO National La aboratory Incorporated Admi inistrative Agency 1952 Radio Research Laboratory 1988 Communications Research Laboratory

Information and Communications Technology 5 6 Major Facilities of NICT Kobe Advanced ICT Research Center (Kobe) Knowledge Creating Communication Research Center, Universal Media Research Center

Center (Yokosuka) 7 Areas of Research and Development in NICT Universal Communication Technology Knowledge creation technologies which enhance the intellectual creativity of individuals Communication

2 アジア ( シンカホールタイ ) へ米国へ The NICT was born by merging CRL and TAO National La aboratory Incorporated Admi inistrative Agency 1952 Radio Research Laboratory 1988 Communications Research Laboratory 2001 Communications Research Laboratory Telecommunications Satellite Corporation, Japan 1992 Telecommunications Advancement Organization Certified Institution National Institute of Information and Communications Technology 5 6 Major Facilities of NICT Kobe Advanced ICT Research Center (Kobe) Knowledge Creating Communication Research Center, Universal Media Research Center (Keihanna) Headquarters, New Generation Network Research Center, Information Security Research Center, Applied Electromagnetic Research Center (Tokyo) New Generation Wireless Communications Research Center (Yokosuka) 7 Areas of Research and Development in NICT Universal Communication Technology Knowledge creation technologies which enhance the intellectual creativity of individuals Communication technologies which transcend the barriers of language, culture and physical capabilities 早晨好! 좋은아침! Selamat pagi! おはよう Super communication technology Super realistic 3D communication technology ICT for safety and security Technologies which ensure safety and security of the ICT networks which are the foundation for social and economic activities Technologies which broadly ensure security by using ICT and realize a safe and secure social environment Upgraded technologies to counter cyber attacks New Generation Network technology Network technologies which maintain and strengthen Japan s global competitiveness in core technologies such as optical and mobile communications The most advanced basic technologies which lead the world s ICT development Establishment of the latest R&D testbed networks R&D of photonic network technology (Optical packet switching) R&D of space and global environment 8 2

Public Services Determination & Supply of Standard Time and Frequency Hagane mtn.

JJY LF station (40 khz, 250m high antenna) Regular Observation of Ionosphere & Space Environment Info.

Wireless Equipment Towards Ultra-Realistic Broadcasting and Communications 9 10 Developments of Television

3 Public Services Determination & Supply of Standard Time and Frequency Hagane mtn. JJY LF station (60 khz, 200m high antenna) Ootakadoya mtn. JJY LF station (40 khz, 250m high antenna) Regular Observation of Ionosphere & Space Environment Info. Service Space weather forecast center Standard site for antenna calibration Type Approval & Calibration of Wireless Equipment Towards Ultra-Realistic Broadcasting and Communications 9 10 Developments of Television Services My Home Theater System Presen nce Home Theater Photograph Monochrome Television Color Television 90-Inch Large Screen Now Time 11 Projector, Audio Amp and HDD Recorder 12 3

4 Next Television Service Presen nce Home Theater Super Hi-vision (UDTV) Ultra (Super Hi-vision) Color Television Monochrome Television Photograph Research in NHK Science and Tech. Research Labs. (NHK STRL) Now time Super Hi-Vision (UDTV) Ultra-high-definition, Wide-screen System with 4320 Scanning Lines) Future TV System with Greater Sensation of Reality Image formats of SDTV,, D-Cinema and Super Hi-Vision SDTV Super Hi-Vision 720 linesx480 pixels 7xPicture Height lines 1920 pixels 30deg pixels 3.0 Picture Height Digital Cinema 4320 Lines (1080 x4) lines 2160 line es 4096 pixels 7680 pixels (1920x4) Video ITU-R Recommendation BT.1769(LSDI:Large Screen Digital Imagery) 15 Standard viewing distance 0.75 Picture Height 100deg. 1.5 Picture Height 55deg. 16 4

Visual field Why do we need 4000 scanning lines?

that of 70mm cine film horizontally and vertically.

Effective visual field SHV 20 deg. 85 deg.

5 Visual field Why do we need 4000 scanning lines? Comparison of resolution Resolution of Super Hi-Vision is more than two times that of 70mm cine film horizontally and vertically. The viewing angle of Super Hi-Vision includes an induced visual field that creates an immersive sensation Digital cinema Super Hi-Vision Induced visual field = Ultimate System Effective visual field SHV 20 deg. 85 deg. er of pixels /frame Numb mm 35mm Standard 10 6 Cine film Standard TV Video 30 deg 100 deg Field frequency(hz) *Pixels for cine film were calculated at 70 pixels / mm 18 Future Home Theater with Super Hi-vision 3DTV

Progress of Television Service Comparisons of 3DTV Methods Presen nce Color Television Monochrome Television Photograph Now 3D TV Super Hi-Vision (UDTV) time Depth

Availability of conventional (Stereoscopic system) systems video.

extremely fine fringr pattern Auto-stereoscopic in most display.

6 Progress of Television Service Comparisons of 3DTV Methods Presen nce Color Television Monochrome Television Photograph Now 3D TV Super Hi-Vision (UDTV) time Depth clues to be reproduced 3DTV Method Basic configuration Features Convergence & Parallax Motion Parallax Accomo- dation Binocular system Comprises two television Availability of conventional (Stereoscopic system) systems video. High picture quality Multi-view system m Spatial image reproducing system Volumetric or multi-layer system Super multiview/ Integral imaging Holography Comprises three or more television systems. Fills a certain volume of space with multiple display screen Comprise of lens array or pinhole array and highresolution image Based on record and reproduction of extremely fine fringr pattern Auto-stereoscopic in most display. Addition of motion parallax Natural depth in a fixed area, phontom image Reproduces light ray Parallax in omni direction Reproduces wave front Needs coherent light Multi-view 3D Display without special glasses Wide and Large screen Super Multi-view 3D display Without having special 3D glasses on, this display gives viewers both remarkable and suitable image experiences depends on view points. Screen Projectors (2)Table-top 3D Display (1)Large Screen 3D Display (3)Handheld Cubic Display

Super Multi-View Display Horizontal light ray goes as it is, and Vertical light ray is difused through the special screen Projector array generates horizontal motion parallax Projectors top top view

7 Super Multi-View Display Horizontal light ray goes as it is, and Vertical light ray is difused through the special screen Projector array generates horizontal motion parallax Projectors top top view side view Wide and Large screen Super Multi-view 3D display Table-top 3D Display -f VisiOn - Side view Viewing position Reproduced 3D Image Bird eye view Viewing position Table plane Table plane Multi-view in horizontal direction 70 Screen Projectors Experimental super multi-view display Video (a) People can discuss watching 3D objects on the table (b)the principle of fvision Video Currently developing 200 inch-class Glasses-Free 3D Display 27 (c)reproduced 3D images from different view points 28 7

Handheld Cubic Display - gcubik- Electronic Holography gcubik s Concept Video Group-sharing of 3D images Graspable virtual objects Glasses-free autostereoscopic display Glazed-showcase like display

lens array is limited around 20~40. Showcase-like watching requires a wider viewing angle 120. Developed two-tiered IP lens array provides 120 viewing angle.

- No use for special glasses or equipment - Has full parallax capability and provides viewers with more depth clues that help them see more natural and realistic 3D images Experimental setup for

8 Handheld Cubic Display - gcubik- Electronic Holography gcubik s Concept Video Group-sharing of 3D images Graspable virtual objects Glasses-free autostereoscopic display Glazed-showcase like display Implementation Approach A cube formed by autostereoscopic displays (Integral Photography: IP) Put virtual 3D objects inside the cube A Conquered Problem Viewing angle of ordinary lenticular or IP lens array is limited around 20~40. Showcase-like watching requires a wider viewing angle 120. Developed two-tiered IP lens array provides 120 viewing angle. gcubik allows viewing from arbitrary directions using our novel lens array. 29 Holography is a ultimate 3D imaging method - The light from an actual object is recorded and reproduced accurately. - No use for special glasses or equipment - Has full parallax capability and provides viewers with more depth clues that help them see more natural and realistic 3D images Experimental setup for electronic holography 30 Realization of electronic holography system conventional holography - film or printed material mainly still images distribution problem 31 [tasks] target viewing-zone-angle expansion undesirable lights elimination image size enlargement colorization live, moving objects reconstruction electronic holography - electronic device moving image presentation real-time transmission problem MIT (1990) insufficient device capability, i.e. resolution -narrow viewing zone (mono-ocular) -low picture quality small image size others -difficult to capture live, moving objects -difficult colorization, etc. 31 Color holography reconstruction system B laser (DPSS:473 nm) G laser (HeNe:543.5 nm) R laser (HeNe:632.8 nm) LCD input panel DVI 16ch. R hologram drive G hologram unit B hologram collimator polarized beam splitter lens f = 300 mm mirror 8Kx4K LCD panel number of pixels 8192(H) X 4320(V) pixel pitch 4.8 µm mirror spatial filter viewing-zone angle 5.6 degree half-mirror (3 mm thickness) lens f = 300 mm Video 32 8

Right-side viewing Bottom Viewing 34 Multi-sensory Interaction System You feel as if a thing is here.

9 A Real-Time Color Holography System for Live Scene Integral 3D imaging system 1. Capture 2. Processing 3. Display Light Lens array Hologram Laser Camera Observer Optical Image Video camera Display (LCD,ect) IP image Hologram Object Reproduced image Upper viewing 33 UDTV Camera Left-side viewing Right-side viewing Bottom Viewing 34 Multi-sensory Interaction System You feel as if a thing is here. 35 Multi-sensory Interaction Multisensory Interaction System Headphone Half-reflected Mirror 3D Image Force Feedback Device You can see, touch, and hear it as if you are interacting with a real 3D object. System Features Acquirement of multisensory information Acquiring 3D structure, texture images and contact sounds from real objects Analysis and modeling Modeling 3D structure and tactile sense Frequency mode analysis of contact sounds Integration and re-creation of multisensory information Generating 3D image, sound, and tactile sense in accordance with the position, the strength and the way of touch (tapping or scraping). Using this system, we are currently investigating human cognitive mechanisms of multisensory integration. 36 9

Multisensory Interaction System for Re-creating 3D Images, Sounds, the Sense of

with Ultra-Realistic Broadcasting 超臨場感通信放送システムの実現を目指して 201x

20mm Hole of shooter This device can control the diffusion of fragrance in time and

37 From HP of the Ministry of Internal Affairs and Communications Understanding

Developing measurement and analysis techniques using brain imaging, psychophysics,

10 Multisensory Interaction System for Re-creating 3D Images, Sounds, the Sense of Touch, and smell Head phone 3D stereo glasses 3D Image of Baloon Future Home Theater with Ultra-Realistic Broadcasting 超臨場感通信放送システムの実現を目指して 201x 年サッカーワールドカップ決勝戦を立体テレビで観戦 Micro Aroma Shooter Half Mirror Force Display 20mm 20mm 20mm Hole of shooter This device can control the diffusion of fragrance in time and space. 37 From HP of the Ministry of Internal Affairs and Communications Understanding human multi-sensory mechanisms Understanding human multi-sensory mechanisms Developing measurement and analysis techniques using brain imaging, psychophysics, and biometric signals, and formulating human evaluation indexes of presence. 39 Brain imaging using fmri when human is viewing wide and 3D image Brain imaging using NIRS when human is smelling aroma oil 40 10

Brain activity measurement system when watching wide view angle and stereoscopic 3D video images Projected Right and left Images Mirror

magnetic shield box Super telephotographic lens 41 42 Data quantity of Future Video systems Bit Rate(bps) Example :50 display, Viewing

2Gpixel, 72Gbps) 10 10 10 9 10 8 10 7 10 4 10 5 NTSC (Integral Imaging n 2 X1.5Gbps)15Tbps (n:seveal hundreds) (Multi-view nx1.

5Gbps) 10 6 10 7 10 8 10 9 10 10 Number of Pixels 43 Estimated Data rate of future video systems (bps) UDTV Multi-view 3DTV 10M 100M 1G

11 Brain activity measurement system when watching wide view angle and stereoscopic 3D video images Projected Right and left Images Mirror Screen Super wide angle eyepiece lens Eye-tracking Measurement Mirror fmri Examinee Data rate of future video systems Projectors in a magnetic shield box Super telephotographic lens Data quantity of Future Video systems Bit Rate(bps) Example :50 display, Viewing angle 30 degree 600Tbps 40 万 X1.5Gbps) 10 3D-TV (Holography) Super Hi-Vision (3.2Gpixel, 72Gbps) NTSC (Integral Imaging n 2 X1.5Gbps)15Tbps (n:seveal hundreds) (Multi-view nx1.5gbps)150gbps D-Cinema(4K) (Streoscopic 2X1.5Gbps)3Gbps (1.5Gbps) Number of Pixels 43 Estimated Data rate of future video systems (bps) UDTV Multi-view 3DTV 10M 100M 1G 10G 100G 1/70-1/100 (MPEG2) 15-20Mbps Compressed 1.5Gbps Baseband Mbps Compressed Integral Imaging Electronics Holography 1/160-1/500 (H.264) 100Mbps-1Gbps(?) 48Gbps Baseband About 1/1000 1~100Gbps(?) 1T 10T 100T 100Gbps-1Tbps Baseband 1/1000-1/ Tbps Baseband 44 11

realistic experience beyond space and time by physically capturing, transmitting, and reproducing

information in order to provide a greater emotional impact and deeper understanding.

medicine Educational applications Technologies and Science to realize Ultra-Realistic communications

12 What is Ultra-Realistic Communications? Ultra-Realistic Communications Forum Super- realism Means of communication that furnish a highly realistic experience beyond space and time by physically capturing, transmitting, and reproducing multi-sensory stimulation and information on sight, hearing, touch, smell, and taste as faithfully as possible. Transcending realism Means of communication that go beyond realism with more than physically transmitted information in order to provide a greater emotional impact and deeper understanding Future Ultra-Realistic Communications Future Broadcasting Tele-work office Tele-shopping Remote medicine Educational applications Technologies and Science to realize Ultra-Realistic communications Three dimensional image (3DTV) Ultra-high definition image Three dimensional sound reproduction Haptic sensor and reproduction Smell sensor and reproduction Multi-modal, cross-modal, and interaction system Investigation of the mechanism about human perception and cognition of "shared reality

13 Ultra-Realistic Communications Forum Organization On March 7, 2007, the URCF was established with the participation of industry, universities, the Ministry of Internal Affairs and Communications (MIC), and the NICT. The URCF promotes the development of technologies relating to HD images, 3D images, hyper-realistic sound, and multi-sensory communications. Objectives Contribute to the advancement and development of technologies relating to HD images, 3D images, hyper-realistic sound, and multi-sensory communications Actively and efficiently promote R & D, verification experiment, standardization, etc. Create the most advanced ultra-realisticrealistic communications Activities Hosting symposia and workshops Verification experiment Standardization Participating in various technical exhibitions Support of various research project Corporation with related consortia and academic societies NICT Universities i i Industry 3D Academic Consortium Societies Etc. Administrative Office General Meeting Chair Person: Prof. Hiroshi Harashima, The University of Tokyo Promotion Committee Secretariat Advisory Committee Planning and Promotion Committee, Technology Development Committee (Members of URCF 194 at the end of November, 2010) What will be possible using Ultra- Realistic Communication Technologies? Summary From promotion video for invitation World Cup Soccer 2018/2022 to Japan

14 Summary NICT is promoting R & D of the technologies to realize 3D image and sound reproduction, and investigation of the mechanism about human perception and cognition of "shared reality. Through these activities, we will be able to enjoy real and natural communications, "shared reality, as if people at great distance are in the same place at the same time. 53 Thank you for your kind attention! 54 14

Studies for Future Broadcasting Services and Basic Technologies

Research Results 3 Studies for Future Broadcasting Services and Basic Technologies OUTLINE 3.1 Super-Surround Audio-Visual Systems With the aim of realizing an ultra high-definition display system with